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PREFACE 


Work  reported  here  was  conducted  by  the  Human  Resources  Research 
Organization  (HumRRO)  under  Contract  No.  DAHC  19-78-C-0024  with  the 
U.S.  Army  Research  Institute.  The  report  covers  approaches  to  develop 
procedures  that  enable  test  developers  to  select  subsets  of  task 
elements  predictive  of  whole  task  performance  and  to  convert  the  sub¬ 
sets  into  efficient  group  tests. 

The  research  was  performed  at  HumRRO  Western  Division,  Radcliff, 
Kentucky,  where  Mr.  William  C.  Osborn  is  the  Office  Director. 

Charlotte  H.  Campbell,  James  H.  Harris,  and  William  C.  Osborn  per¬ 
formed  the  work  reported. 
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SUMMARY 


This  report  describes  approaches  taken  to  develop  procedures 
that  enable  test  developers  to  identify  elements  of  tasks  that  pre¬ 
dict  overall  task  performance. 


REQUIREMENT 


The  requirement  to  which  this  work  was  addressed  was  to  develop 
an  economical  method  for  drawing  valid  inferences  about  a  soldier's 
ability  to  perform  the  tasks  on  which  he  has  been  trained.  The 
project  had  two  objectives.  The  first  objective,  related  to  the  key 
problem  in  developing  economical  proficiency  tests,  was  to  develop 
procedures  for  identity ing  elements  of  tasks  that  predict  overall 
task  performance.  The  second  objective,  predicated  on  the  success 
of  systematically  identifying  the  predictive  elements,  related  to 
developing  a  technique  for  constructing  reliable,  valid,  feasible, 
and  acceptable  hands-on  tests  for  the  subsets.  The  specific  objective 
was  to  develop  procedures  for  constructing  "less  than  full"  hands- 
on  tests  that  yield  individual  scores.  ^ 


PROCEDURES  AND  RESULTS 


Three  approaches  were  taken  to  meet  the  first  objective  of  the 
project;  that  is,  selecting  subsets  that  predict  overall  task  perform¬ 
ance.  The  first  approach  follows  closely  the  task  analysis  review 
procedure  developed  for  the  skill  qualification  test  workshops,  with 
changes  made  to  reflect  group  or  expensive  individual  tasks.  The 
intent  was  for  subject  matter  experts  to  identify  task  elements  that 
were  likely  candidates  for  testing  and  then  verify  the  identifications 
by  having  soldiers  perform  the  tasks  reviewed  and  comparing  their 
performance  with  the  predictions.  For  a  variety  of  reasons,  the 
verification  did  not  proceed  as  planned.  Essentially,  each  task  took 
so  long  to  perform,  at  least  for  soldiers  at  the  entry-level  of 
experience,  that  there  was  neither  the  time  nor  the  equipment  avail¬ 
able  to  conduct  the  verification.  For  this  reason,  a  second  approach 
was  taken  to  develop  procedures  that  would  enable  test  developers  to 
identify  predictive  elements  systematically. 

A  set  of  performance  data  from  an  Army  training  study  (ARTS) 
was  used  to  verify  selection  procedures.  The  plan  was  to  develop  a 
selection  instrument  and  have  subject  matter  experts  select  elements 
to  test.  If  the  inter-judge  agreement  levels  were  satisfactory,  the 
selections  could  be  compared  with  available  performance  results  on 
the  large  sample  of  soldiers.  In  order  for  the  selection  procedures 
to  be  systematic  and  useful  for  test  developers,  they  must  work  as 
well  with  one  subject  matter  expert  as  with  another.  High  agreement 


v 


would  indicate  that  having  one  person  select  elements  would  be  as 
valid  as  collecting  the  same  information  from  any  other,  or  from  a 
group,  with  a  considerable  savings  in  time  and  effort.  Inter-judge 
agreement  levels  were  too  low  to  warrant  use  of  the  selection  pro¬ 
cedures  in  test  development. 

In  the  third  approach,  task  performance  data  were  examined  to 
determine  empirically  the  most  likely  testing  point (s)  within  each 
task.  Based  on  the  results  of  the  empirical  examination,  procedures 
could  be  established  for  test  developers  to  select  these  same  testing 
points  without  benefit  of  test  results.  A  forward  (stepwise)  mul¬ 
tiple  regression  and  a  Guttman  scalopram  analysis  were  performed  on 
the  data  to  determine  if  a)  a  subset  of  elements  could  be  identified 
for  each  task  which  were  predictive  of  whole  task  performance,  and 
b)  the  elements  comprising  a  task  were  unidimensional  and  of  incre¬ 
mental  difficulty.  The  results  of  the  statistical  analyses  indicate 
that  the  most  predictive  elements  could  be  identified  empirically. 

The  problem  is  that  the  most  predictive  elements  in  the  array,' 
whether  evaluated  within  each  task,  over  all  tasks,  or  within  a  task 
category,  seem  to  have  nothing  in  common. 

It  was  recommended  that  future  approaches  to  identifying  and 
classifying  predictive  task  elements  be  based  on  experimentally 
generated  performance  data  supported  by  comprehensive  diagnostic 
scoring.  It  was  noted,  however,  that  if  underlying  sources  of  task 
element  commonality  are  found,  they  may  be  of  no  use  in  test  develop- 
memt.  For  example,  the  most-predictive  elements  may  be  those  that 
are  not  covered  in  training,  or  those  that  cannot  be  observed,  or 
those  that  are  the  least  well  described  in  the  job  aid. 

USE  OF  FINDINGS 


Since  results  of  this  work  indicate  that  extreme  caution  should 
be  exercised  in  attempting  to  develop  a  performance  test  covering  a 
sample  of  task  elements  only,  the  report  should  be  of  interest  to 
test  developers  throughout  the  Army's  service  schools. 
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AN  ATTEMPT  TO  IDENTIFY  INDICATORS  OF  COMPETENCE  ON  MECHANICAL 
MAINTENANCE  TASKS 


CHAPTER  1 
INTRODUCTION 


BACKGROUND 


One  responsibility  of  Army  service  schools  is  to  certify  trainee 
proficiency.  The  developer  of  training  certification  tests  shares 
the  dilemma  of  those  involved  in  all  applications  of  proficiency 
measurement:  achieving  a  balance  between  test  quality  and  adminis¬ 

tration  economy.*  To  achieve  some  perspective  on  the  problem  .of 
balancing  quality  and  economy,  consider  first,  that  a  full  perform¬ 
ance  test  (demonstration  of  the  actual  criterion  behavior  in  a 
realistic  criterion  setting)  is  the  most  valid  type  of  proficiency 
test,  and  second,  that  a  group  administerable  test  yielding  scorable 
task  product  and  process  scores  for  individuals  is  the  most  feasible 
type  of  proficiency  test.  Any  test,  then,  that  is  both  full  perform¬ 
ance  and  group  administerable — valid  and  feasible — is  an  efficient 
test  and  should  present  no  problem  to  the  developer  or  administrator. 
The  balance  is  difficult  to  achieve  at  most  Army  service  schools 
where  budgetary  constraints  force  test  developers  to  confront  the 
problems  of  equipment  availability,  scorer  qualifications,  and  time 
limitations . 

These  confrontations  cause  severe  problems  for  the  schools' 
quality  control  systems.  Each  school  has  a  quality  control  system 
based  on  comprehensive  performance  tests.  Since  the  most  clearly 
valid  form  of  performance  test  is  the  hands-on  test,  service  schools 
have  constructed  hands-on  tests  for  most  MOS  producing  courses.  The 
tests  have  two  purposes:  to  provide  feedback  on  the  quality  of 
training  and  to  certify  whether  trainees  are  able  to  perform  the 
tasks  addressed  in  training. 

The  most  common  problem  with  this  approach  is  that  there  are 
too  many  tasks  to  conduct  a  hands-on  test  for  each  trainee  on  each 
task.  Trainers  compromise  by  testing  a  sample  of  tasks.  The  sample 
consists  of  the  most  "critical"  tasks,  which  are  tested  each  cycle, 
and  a  random  sample  of  the  remaining  tasks.  This  compromise  reduces 
the  ability  to  certify  trainees  on  the  complete  job,  but  if  all 
tasks  are  eventually  tested,  retains  the  feedback  to  trainers  on  the 
quality  of  training.  At  least  it  appears  to  in  theory. 


*  Osborn ,  W.C.  An  Approach  to  the  Development  of  Synthetic  Performance 
Tests  for  Use  in  Training  Evaluation.  HumRRO  Professional  Paper 
30-70,  December  1970. 


In  reality,  many  schools  do  not  vary  the  tasks  tested  in  such 
a  way  to  assure  that  all  tasks  are  eventually  tested.  Varying  the 
sample  entails  requesting  new  equipment,  modifying  the  test  area, 
formalizing  the  test  instrument,  and  training  new  scorers.  The  first 
sample  tends  to  be  the  only  sample. 

Trainers  Lnus  must  draw  inferences  from  a  sample  of  behavior 
to  evaluate  a  larger  class  of  behavior.  But  since  the  tasks  are  not 
selected  systematically,  many  service  school  test  programs  are 
sufficient  neither  for  training  feedback  nor  for  individual  certi¬ 
fication.  In  this  way,  test  quality  and  administration  economy  get 
out  of  balance. 

If  the  schools'  quality  control  systems  are  to  be  credible, 
they  must  either  reduce  the  scope  of  training  or  confront  the  issue 
of  administration  economy.  Since  most  schools  will  still  train  too 
many  tasks  to  conduct  a  hands-on  test  for  each  soldier  on  each  task, 
some  compromises  will  be  required  to  maintain  an  acceptable  balance 
between  test  quality  and  administration  economy.  The  purpose  of 
the  work  reported  here  was  to  provide  a  basis  for  compromises  to 
maintain  the  balance  for  at  least  some  of  the  tasks  taught  at  service 
schools . 


PROBLEM 


One  aspect  of  performance  testing  that  is  important  at  some 
service  schools  is  testing  tasks  in  student  groups.  Hands-on  tests 
for  groups  cover  two  types  of  tasks: 

1.  Group  tasks  which  are  normally,  and  often 
necessarily,  performed  on  the  job  by  a 
team  or  crew.  This  type  of  task  is  rarely 
addressed  in  service  school  training 
testing . 

2.  Individual  tasks  which  require  so  much  time 
or  equipment  that  the  only  economical  method 
of  testing  is  with  a  group  of  soldiers. 

Each  soldier  performs  only  a  part  of  the 
task,  working  simultaneously  with  others  on 
different  components  of  the  equipment,  or  in 
circuit  on  the  same  part  of  the  equipment. 

Strictly  speaking,  these  are  not  group  tasks, 
but  the  requirements  for  training  and  testing 
are  so  extensive  that  in  the  school  setting 
they  are  performed  by  groups. 
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If  tests  were  needed  only  to  provide  feedback  on  the  quality 
of  training,  group  testing  would  pose  only  minor  problems.  Trainers 
could  simply  test  examinees  as  a  group  and  draw  inferences  concerning 
full  task  training  from  the  performance  of  the  group  as  a  whole. 

Unless  the  part-task  performed  by  an  individual  is  representative 
of  the  whole  task  however,  part-task  test  results  are  not  suitable 
for  certifying  individual  trainees  as  proficient  on  the  whole  task. 

The  approach  of  group  testing  on  random  part-tasks  is  a  microcosm  of 
end-of-cycle  tests — trainers  must  draw  inferences  from  a  sample  of 
behavior  (part-tasks)  to  evaluate  a  larger  class  of  behavior  (whole 
tasks)  even  though  the  sample  is  not  selected  systematically. 

Trainers  have  another  alternative  for  group  testing  that  increases 
both  the  quality  and  cost  of  the  test.  This  alternative  is  to  test 
each  soldier  in  each  duty  position.  For  example,  if  one  soldier 
operates  a  hoist  while  a  second  guides  a  power  pack  into  position  and 
a  third  soldier  connects  the  pack,  trainers  could  test  the  group  three 
times,  rotating  soldiers  through  each  position.  The  information  would 
be  better  than  a  test  of  random  samples  of  part-tasks,  but  the  increase 
in  cost  would  outweigh  the  increase  in.  quality  to  the  point  where 
there  is  no  savings  over  a  full  task  test  for  each  soldier.  Even  if 
service  schools  did  not  face  major  time,  personnel,  and  equipment 
constraints,  the  improvement  in  test  quality  is  more  apparent  than 
real.  The  validity  of  this  test  approach  is  reduced  by  two  factors 
that  compromise  test  standardization: 

1.  The  learning  effect  that  occurs  for  part-task  X 
while  performing  part-task  Y  is  not  controlled. 

2.  Since  the  quality  of  performance  of  others  in 
the  group  is  not  controlled,  a  given  soldier's 
proficiency  is  subject  to  measurement  error 
resulting  from  unstandardized  test  conditions . 

Not  all  Array  service  schools  train  tasks  which  require  group 
testing.  But  when  a  school  does  address  this  type  of  group  task,  the 
tasks  are  usually  at  the  heart  of  the  school's  training  mission. 

If  they  are  to  have  an  efficient  quality  control  system,  they  need 
an  economical  method  to  certify  trainee  proficiency  on  such  group 
tasks . 


A  POSSIBLE  SOLUTION 


Individual  hands-on  tests,  even  part-task  tests,  are  usually 
easier  to  score  and  more  obviously  fair  than  group  tests  since  everyone 
faces  the  same  requirements.  But  a  group  test  that  results  in  indi¬ 
vidual  scores  is  the  most  efficient  method  for  testing  expensive 
individual  tasks.  Thus,  a  likely  approach  would  be  to  develop  "less 
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than  full-performance"  tests  on  a  sample  of  a  trainee's  task  performance. 
Since  a  sample  of  performance  was  proposed,  the  most  efficient  sample 
would  be  one  that  included  only  those  task  elements  that  are  the  most 
predictive  of  whole  task  proficiency.  Each  soldier  would  perform  a 
subset  of  the  task  and  his  performance  on  the  assigned  subset  would  be 
the  basis  for  his  score.  The  tests  would  have  to  meet  three  criteria: 

1.  Each  examinee  would  be  required  to  demonstrate 
the  same  types  of  behaviors  as  would  be  required 
on  a  full  performance  test. 

2.  The  tests  could  be  administered  to  a  group  of 
examinees . 

3.  The  tests  would  result  in  individual  scores. 

The  success  of  this  approach  hinged  on  developing  a  methodology 
to  identify  elements  of  tasks  predictive  of  overall  task  performance. 

If  predictive  elements  could  be  identified  systematically,  procedures 
could  be  developed  to  enable  Army  service  school  test  developers  to 
select  the  elements  and  convert  them  into  efficient  group  tests. 


OBJECTIVE 


The  project  originally  had  two  objectives.  The  first  related  to 
the  key  problem  in  developing  "less  than  full-performance"  tests  that 
yield  individual  scores.  The  objective  was: 

.  To  develop  procedures  for  identifying  elements 
of  tasks  that  predict  overall  task  performance. 

The  second  objective,  predicated  on  the  success  of  systematically 
identifying  the  predictive  elements,  related  to  developing  a  technique 
for  constructing  reliable,  valid,  feasible,  and  acceptable  hands-on 
tests  for  the  subsets.  The  specific  objective  was: 

.  To  develop  procedures  for  constructing  "less 
than  full-performance"  hands-on  tests  that 
yield  individual  scores. 

As  indicated,  work  on  the  second  objective  depended  on  accomplishing 
the  first.  And  since,  despite  our  efforts,  procedures  for  identifying 
predictive  task  elements  could  not  be  validated,  there  was  no  basis  for 
undertaking  development  of  test  construction  guidance  to  use  the  procedures. 
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OVERVIEW 


This  report  documents  attempts  to  identify  elements  of  mechanical 
maintenance  tasks  predictive  of  overall  task  performance.  The 
research  evolved — largely  because  of  practical  limitations  on  data 
collection — into  three  phases.  These  phases  are  reported  as  Studies  1, 
II,  and  III. 

The  primary  purpose  of  the  project  was  to  develop  a  set  of  pro¬ 
cedures  which  would  enable  test  developers  to  identify  task  elements 
that  predict  successful  task  performance.  Since  test  development 
normally  begins  with  a  review  of  the  task  analysis  data,  the  most 
logical  approach  was  to  tie  the  identification  procedures  into  the 
task  analysis  review  phase.  Because  task  analysis  data  must  often  be 
modified  by  the  test  developer  when  the  data  are  to  be  used  as  a  basis 
for  test  construction,  no  new  step  would  be  added  to  the  test  develop¬ 
ment  cycle;  changes  would  be  made  to  an  existing  one.  Therefore,  in 
Study  I  procedures  were  developed  by  which  test  developers  would  have 
subject  matter  experts  (SME)  review  and  modify  task  analysis  data. 

The  procedures  were  to  serve  two  purposes.  First,  they  would  insure 
the  task  analysis  data  were  at  a  level  of  detail  sufficient  to  be 
useful  for  test  construction.  Second,  the  procedures  would  enable 
test  developers  to  identify  task  elements  that  were  likely  candidates 
for  testing;  that  is,  those  elements  whose  successful  performance 
predicts  overall  task  performance.  These  identifications  were  to  be 
verified  by  having  soldiers  perform  the  tasks  reviewed  and  comparing 
their  performance  with  test  developer  predictions. 

Since  the  data  from  Study  I  did  not  confirm  the  usefulness  of 
task  analytic  data  in  revealing  predictive  elements  for  part-task 
testing,  another  approach  was  tried  in  Study  II  in  which  subject 
matter  experts  directly  nominated  task  elements  for  testing  according 
to  prescribed  criteria.  Results  of  the  second  study  were  also  incon¬ 
clusive.  In  Study  III  the  judgment  of  subject  matter  experts  was 
circumvented  altogether  and  task  elements  were  explored  empirically 
in  an  effort  to  identify  from  actual  test  results  those  elements 
predictive  of  whole  task  performance. 

Data  in  all  studies  pertained  to  the  heavy  equipment  maintenance 
field — 63H10  automotive  maintenance,  45L10  artillery  maintenance, 
and  63C10  tracked  vehicle  maintenance — since  job  tasks  in  this  field 
are  typically  difficult  to  test  efficiently. 

The  method,  results,  and  discussion  of  each  of  the  three  studies 
follow.  A  discussion  of  the  research  in  terms  of  "lessons  learned" 
and  implications  for  future  work  concludes  the  report. 


CHAPTER  2 


STUDY  I:  IDENTIFYING  PREDICTIVE  ELEMENTS 
FROM  SME  TASK  ANALYSIS  REVIEW 


The  first  study  was  conducted  according  to  the  original  plan  of 
research.  Its  intent  was  to  develop  and  validate  procedures  for 
test  developers  to  review  and  modify  task  analysis  data  for  use  in 
part-task  development.  The  approach  was  based  on  task  analysis  review 
procedures  for  SQT  development  with  changes  made  to  reflect  the  con¬ 
cern  of  the  research  with  group  or  expensive  individual  tasks. 


METHOD 


Two  activities  were  performed  to  develop  a  procedure  that  test 
developers  could  use  to  review  and  modify  task  analysis  data.  The  two 
activities  were  1)  development  of  task  analysis  review  and  modification 
procedures,  and  2)  conduct  of  task  analysis  review. 


Task  Analysis  Review  and  Modification  Procedures 

Test  developers  must  follow  a  logical  review  process  in  order  to 
identify,  as  early  as  possible,  information  gaps  in  the  existing  task 
analysis  data. 1  The  procedure  consists  of  seven  steps  or  items  to 
provide  test  construction  information.  The  questionnaire  developed 
for  task  analysis  review  and  modification  is  presented  at  Appendix  A. 

Because  mechanical  maintenance  tasks  contain  so  many  performance 
elements,  the  procedure  begins  with  an  item  to  reduce  the  number  of 
elements  a  reviewer  has  to  consider  during  the  review  of  any  task. 
Item  1  enables  the  reviewer  to  identify  any  subtasks  within  the  task 
being  reviewed  and  then  to  continue  the  review  procedure  for  each  of 
the  subtasks.  For  example,  eight  subtasks  were  identified  for  the 
task,  "Replace  transmission  5-ton,  M813."  The  eight  subtasks  are: 


1. 

Disconnect  power  take-off  shaft 

and 

PTO  linkage 

2. 

Remove  transmission. 

3. 

Remove  clutch  assembly. 

4. 

Test  and  adjust  clutch  assembly 
pressure  plate  for  warpage. 

and 

inspect 

5. 

Install  clutch  assembly. 

^Campbell,  R.C.,  Ford,  P.,  and  Campbell,  C.H.  Development  of  a  Morksho 
on  Construction  and  Validation  of  Skill  Qualification  Tests.  HumRRO 
Final  Report  FR-WD(KY)-78-2 ,  March  1978. 
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6.  Install  transmission. 

7.  Connect  power  take-off  shaft  and  PTO  linkage. 

8.  Adjust  clutch  linkage  and  free  travel. 

Each  subtask  defines  an  action  that  has  a  measurable  outcome.  The 
instructions  tell  the  reviewer  to  continue  the  review  procedure 
with  the  first  subtask,  then  repeat  the  review  procedure  for  the 
remaining  subtasks  identified  in  Item  1. 

The  purpose  of  Item  2  is  to  determine  if  the  subtask  might  be 
performed  under  different  conditions  and  if  so,  whether  the  subtask 
is  then  performed  differently.  Changed  conditions  sometimes  alter 
the  elements  in  task  performance  and  sometimes  make  the  task  more 
or  less  difficult.  For  example,  removing  the  transmission  on  a 
5-ton,  M813  is  more  difficult  when  done  outdoors  in  the  rain  than 
when  done  in  a  maintenance  shop. 

The  third  item  is  necessary  to  account  for  the  inclusion  of  all 
task  elements.  Here,  the  reviewer  adds  elements  to  permit  identi¬ 
fication  of  correct  performance,  deletes  elements  not  needed  to 
identify  correct  performance,  or  revises  elements  to  permit  ident- 
fication  of  correct  performance  by  defining  observable  actions. 

The  fourth  and  fifth  items  begin  the  identification  of  task 
elements  that  are  likely  candidates  for  testing.  Reviewers  are  first 
asked  to  identify  the  most  common  sources  of  failure  in  subtask  per¬ 
formance.  The  most  common  mistakes  in  a  procedure  are  usually  the 
best  predictors  of  overall  performance.  The  fifth  item  addresses 
the  issue  of  criticality.  The  intent  of  the  item  is  to  assure  that 
the  most  important  elements  are  tested  even  though  they  may  not  be 
sources  of  frequent  errors.  Here,  most  safety  procedures,  elements 
that  can  cause  the  system  to  fail,  and  elements  that  are  not  detected 
by  checks  in  the  system,  are  included.  Although  these  elements  may 
not  be  as  predictive  as  frequent  sources  of  error,  they  must  be 
included  for  the  test  to  be  a  credible  check  on  the  quality  of  training. 

The  necessity  of  performing  any  of  the  task  elements  in  sequence 
is  determined  in  Item  6.  This  does  not  include  elements  which  are 
sequential  because  of  equipment  design;  for  example,  on  the  5-ton, 

M813,  the  transmission  must  be  removed  before  the  clutch  assembly  can 
be  removed.  It  does  include,  for  the  task,  "Replace  5-ton,  M813 
transmission,"  performing  all  the  elements  regarding  connecting  heavy 
lifting  device  and  absorbing  weight  of  transmission  before  removing 
last  two  capscrews  and  lockwashers  securing  clutch  housing  to  flywheel 
housing . 

The  final  item  (Item  7)  identifies  time  constraints  for  any 
element(s)  and  the  consequences  of  failing  to  perform  the  element (s) 
within  that  time. 
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Task  Analysis  Review 

Three  tasks  were  selected  for  task  analysis  review  and  modification. 
The  three  tasks  were: 

1.  Borescope  and  pullover  gage  cannon  tubes 
(MOS  45L10,  Artillery  Repair). 

2.  Inspect  declutching  feed  mecahnism 
(MOS  45L10,  Artillery  Repair). 

3.  Replace  transmission,  5-ton,  M813 
(MOS  63H10,  Automotive  Repair). 

These  tasks  were  selected  because  they  were  each  group  trained  and 
group  tested,  and  they  each  took  more  than  one  hour  to  perform. 

The  task  analysis  review  was  conducted  by  members  of  the  project 
staff  with  SME  from  the  Ordnance  School.  Four  SME  reviewed  each 
task.  For  the  two  45L10  tasks,  the  SME  were  two  civilian  instructors 
and  two  E-7  instructors.  The  review  procedure  for  each  of  the  two 
45L10  Artillery  Repair  tasks  was  as  follows: 

.  The  SME  panel  was  convened  and  each  item  in  the 
questionnaire  (Appendix  A)  was  discussed  with 
each  SME.  All  SME  responses  were  recorded.  The 
panel  format  was  used  to  have  more  control  over 
the  review  during  the  initial  tryout. 

.  The  individual  SME  responses  were  discussed  and 
consensus  answers  reached  for  each  item. 

.  The  task  was  performed  in  its  entirety  by  a  member 
of  the  project  staff.  The  task  was  performed  for 
three  reasons:  first,  to  enable  the  staff  member 
to  learn  how  to  perform  the  task;  second,  to  assess 
the  adequacy  of  the  reference  materials  available 
to  the  soldier  when  he  performs  the  task;  and, 
third,  to  assess  the  adequacy  of  the  task  analysis. 

The  63H10  task  was  reviewed  by  three  people  from  the  mobility 
branch  of  the  Ordnance  School;  one  was  the  Branch  Chief  (a  warrant 
officer),  one  was  an  E-6  instructor,  and  one  was  an  E-5  instructor. 

The  fourth  63H10  reviewer  was  an  E-6  from  the  task  analysis  branch 
of  the  Directorate  of  Training.  The  procedure  for  reviewing  the 
63H10  automotive  repair  task  was  reversed;  that  is,  a  member  of  the 
project  staff  performed  the  task  in  its  entirety  and  then  discussed 
the  Task  Analysis  Review  and  Modification  questionnaire  with  the 
four  SME.  The  procedure  was  reversed  to  make  the  discussion  of  the 
task  more  beneficial  since  the  "test  developer"  (in  this  case,  a 
member  of  the  project  staff)  would  be  more  familiar  with  the  task. 
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RESULTS 


The  results  of  the  review  and  modification  are  presented 
(Tables  1,  2,  and  3)  for  selected  items  on  the  questionnaire. 
Additionally,  the  complete  revised  task  analysis  for  the  63H10 
automotive  repair  task  is  presented  at  Appendix  B.  The  subtasks 
identified  for  each  task  are  listed  in  Table  1.  Each  subtask 
can  stand  on  its  own  as  a  separate  task  with  a  measurable  outcome. 
Changed  conditions  do  not  alter  the  elements  for  any  of  the  three 
tasks  (Item  2  in  the  Task  Analysis  Review  and  Modification).  The 
ease  with  which  parts  are  removed  and  replaced,  of  course,  varies 
with  the  condition  (e.g.,  rusty,  not  rusty;  new,  old)  of  the  parts. 

The  common  sources  of  error  in  tasks  are  shown  in  Table  2. 

These  are  the  elements  which  panel  members  indicated  are  most  often 
performed  wrong  or  not  performed;  they  make  up  what  would  be  the 
most  difficult  items.  Included  among  the  "difficult"  items  are 
such  elements  as: 

.  Select  wrong  pullover  gage  stop  (Borescope 
and  pullover  gage  cannon  tubes) . 

.  Put  declutching  feed  mechanism  torsion  spring 
in  backwards  (Inspect  declutching  feed 
mechanism) . 

.  Leave  shipping  bolts  in  5-ton,  M813  clutch 
pressure  plate  assembly  (Replace  trans¬ 
mission,  5-ton,  M813). 

The  elements  which  can  result  in  serious  consequences  if  not 
performed  correctly  (or  not  performed)  are  given  in  Table  3.  The 
intent,  again,  was  to  assure  that  the  most  critical  elements  emerged 
as  candidates  for  testing.  Critical  elements  include: 

.  Identify  defects  which  condemn  cannon  tube 
(Borescope  and  pullover  gage  cannon  tubes). 

.  Time  gun  to  declutching  feeder  (Inspect 
declutching  feed  mechanism) . 

.  Adjust  5-ton,  M813  clutch  linkage  (Replace 
transmission,  5-ton,  M813) . 

On  the  revised  task  analysis  for  the  task,  "Replace  transmission" 
(Appendix  B) ,  the  critical  and  difficult  elements  are  marked  with  an 
asterisk  (*) .  Those  elements  which  must  be  performed  in  sequence 
for  reasons  other  than  equipment  design  are  indicated  in  the  task 
analysis  (Appendix  B)  by  a  double  asterisk  (**) .  There  were  no  time 
constraints  for  performing  any  element (s)  among  the  three  tasks 
reviewed . 
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Table  1 


Subtasks  Identified  for  Each  Task  Reviewed  and  Modified 

(Item  1) 


Task 


Subtasks 


45L10  Borescope  and  pullover  1. 

gage  cannon  tubes  2 

3. 

4. 

5. 

6. 

7. 

45L10  Inspect  declutching  1. 

feed  mechanism  2 

3. 

4. 

5. 

6. 

7. 

8. 

63H10  Replace  transmission,  1. 

5-ton,  M813 

2. 

3. 

4. 

5. 

6. 

7. 

8. 


Set  up  borescope. 

Borescope  cannon  tube. 

Take  down  borescope. 

Determine  if  cannon  tube  is 
required  to  be  pullover  gaged. 

Set  up  pullover  gage. 

Pullover  gage  cannon  tube. 

Complete  DA  Forms  2404,  2407, 

2408-4. 

Remove  mechanism  from  20mm  cannon. 

Inspect  mechanism  before  dis¬ 
assembly. 

Disassemble  mechanism. 

Clean  mechanism. 

Inspect  mechanism  after  dis¬ 
assembly  . 

Assemble  and  time  mechanism. 

Test  mechanism  for  wear  and  warp 
(operational  check) . 

Install  mechanism  on  20mm  cannon. 

Disconnect  power  take-off  shaft 
and  PTO  linkage. 

Remove  transmission. 

Remove  clutch  assembly. 

Test  and  adjust  clutch  assembly 
and  inspect  for  warpage. 

Install  clutch  assembly. 

Install  transmission. 

Connect  power  take-off  shaft  and 
PTO  linkage. 

Adjust  clutch  linkage  and  free  travel 
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Table-  2 


Common  Sources  of  Error  (Host  Difficult  Elements)  in  Tasks 

(Item  4) 


Number  of 

Number  of  Common  Sources  Examples  of 


Task 

Elements 

of  Error 

Common  Sources  of  Error 

45L10  Borescope  and 

66 

10 

.  Line  up  reference  line 

pullover  gage 

with  illuminating  head 

cannon  tubes 

mirror . 

*.  Identify  defects  which 
condemn  cannon  tube. 


Select  wrong  pullover 
gage  stop. 

Stop  pullover  gage 
before  reaching  breech 
face  of  cannon  tube. 


43L10  Inspect  98 

declutching 
feed  mechanism 


5  .  Remove  end  play  of  drive 

shaf  t 

.  Depress  actuating  shaft 
when  installing  and 
timing  end  drive  assembly. 

.  Put  torsion  spring  in 
backwards 


63H10  Replace 

transmission, 
5-ton,  M813 


79 


*Also  identified  as  a  critical  element. 


Leave  shipping  bolts  in 
clutch  pressure  plate 
assembly . 

Leave  grease  on  face  of 
clutch  pressure  plate 
assembly . 

Put  lubricant  in  transmission. 


I 
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Table  3 


Elements  Which  Can  Result  in  Serious  Consequences 
If  Not  Performed  Correctly 

(Item  5) 


TASK:  Borescope  and  pullover  gage  cannon  tubes  (45L10) 

ELEMENTS:  *.  Identify  defects  which  condemn  cannon  tubes. 

.  Measure  bore  width  correctly. 

.  Read  bore  width  measurement  correctly. 

.  Select  appropriate  Table  in  TM  9-4933-200-35. 


TASK:  Inspect  declutching  feed  mechanism,  20mm  M163  (45L10) 
ELEMENTS:  .  Time  gun  to  declutching  feeder. 


TASK:  Replace  transmission,  5-ton,  M813  (63H10) 

ELEMENTS:  .  Reverse  transmission  vent  lines. 

.  Adjust  clutch  linkage  and  free  travel. 

.  Put  lubricant  in  transmission. 

.  Put  too  much  lubricant  in  transmission. 
.  Adjust  pressure  plate  release  levers. 

.  Adjust  pressure  plate. 


*Also  identified  as  a  difficult  element. 


DISCUSSION 


The  results  of  the  task  analysis  review  and  modification  for 
each  task  indicate  that  the  procedures  can  be  used  to  provide  task 
performance  information  at  a  level  of  detail  sufficient  for  test 
construction.  All  elements  required  for  successful  task  performance 
were  included  in  the  revised  task  analysis. 

The  reader  will  recall  that  the  second  purpose  of  the  review  and 
modification  procedures  was  to  enable  test  developers  to  identify 
task  elements  whose  successful  performance  predicts  overall  task 
performance.  The  SME  selection  of  difficult  elements  for  each  task 
was  to  be  verified  by  testing  a  group  of  soldiers  who  had  been  trained 
on  the  task  and  comparing  their  performance  on  each  element  with  SHE 
predictions  of  the  most  difficult  element(s).  For  a  variety  of 
reasons,  the  verification  did  not  proceed  as  planned.  Essentially, 
each  task  took  so  long  to  perform,  at  least  for  soldiers  at  the 
entry-level  of  experience,  that  there  was  neither  the  time  nor  the 
equipment  available  to  conduct  the  verification.  For  this  reason, 
other  ways  were  considered  to  verify  a  set  of  procedures  that  would 
enable  test  developers  to  identify  predictive  elements  systematically. 
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CHAPTER  3 


STUDY  II:  IDENTIFYING  PREDICTIVE  ELEMENTS 
DIRECTLY  FROM  SME  JUDGMENTS 


Test  results  from  a  world-wide  Army  Training  Study  (ARTS) 
appeared  promising  as  a  set  of  data  for  studying  the  validity  of 
element  selection  procedures.  Included  in  these  data  were  six 
tasks  from  the  63H10  automotive  mechanic  and  six  tasks  from  the 
63C10  tracked  vehicle  MOS.  The  plan  was  to  have  SME  select  directly 
elements  to  test  rather  than  the  more  indirect  approach  of  rating 
elements  for  difficulty  and  criticality.  If  SME  agreement  levels 
were  satisfactory,  the  SME  selections  could  be  compared  with  available 
performance  results  on  the  large  sample  of  soldiers.  The  tasks  used 
in  the  ARTS  are  considerably  shorter  in  terms  of  number  of  elements 
and  time  to  perform  than  are  the  three  original  tasks.  However, 
if  the  procedures  could  be  verified  on  short  tasks,  they  would  work 
on  long  tasks,  which  may  be  viewed  essentially  as  a  collection  of 
short  tasks. 


METHOD 


This  second  procedure  for  task  analysis  review  and  modification 
focused  exclusively  on  the  second  purpose  (identifying  predictive 
elements)  for  an  SME  review  of  task  analysis.  The  first  purpose  of 
an  SME  review  procedure  (to  provide  task  performance  information  at 
a  level  of  detail  sufficient  to  be  useful  for  test  construction)  was 
met  by  the  initial  procedures  (Appendix  A).  Therefore,  project  staff 
conducted  the  task  analysis  review  for  this  purpose.  In  the  course 
of  this  review,  three  tasks  (63H,  Tasks  2,  3,  and  3)  were  modified. 

The  study  entailed  the  development  and  administration  of  a  question¬ 
naire  to  elicit  SME  judgments  about  priority  task  elements  for 
testing.  Performance  on  these  elements  was  then  to  be  rescored  by 
task  for  soldiers  tested  in  ARTS,  and  the  "part-task"  scores  correlated 
with  whole  task  performance.  Low  agreement  among  SME  ratings, 
unfortunately,  precluded  the  planned  analysis.  Questionnaire  prepara¬ 
tion  and  administration  were  as  follows. 


Task  Element  Selection  Questionnaire 


The  selection  questionnaire  contained  twelve  questions.  Four 
questions  concerned  selecting  a  specific  number  of  task  elements 
(from  one  to  four)  which,  if  the  soldier  performed  successfully,  would 
convince  the  SME  that  the  soldier  could  perform  the  entire  task  suc¬ 
cessfully.  The  four  questions  were: 


1. 


You  want 
There  is 
elements 
you  want 


to  know  if  a  soldier  can  do  this  task, 
time  for  him  to  do  only  one  of  the 
for  the  task.  Which  one  element  would 
to  see  him  do? 


2.  You  want  to  know  if  a  soldier  can  do  this  task. 
There  is  time  for  him  to  do  only  two  of  the 
elements  for  the  task.  Which  two  elements 
would  you  want  to  see  him  do? 


> 


3.  You  want  to  know  if  a  soldier  can  do  this  task. 
There  is  time  for  him  to  do  only  three  of  the 
elements  for  the  task.  Which  three  elements  •' 
would  you  want  to  see  him  do? 

4.  You  want  to  know  if  a  soldier  can  do  this  ta^k. 
There  is  time  for  him  to  do  only  four  of  the 
elements  for  the  task.  Which  four  elements 
would  you  want  to  see  him  do? 


Four  questions  asked  SME  to  select  elements  to  test  given  four 
time  periods  for  testing.  The  four  questions  were: 

5.  You  want  to  know  if  a  soldier  can  do  this  task. 

You  only  have  five  minutes  for  the  J,est.  Which 
element  or  elements  would  you  want  to  see  him 
do? 


<« 


6.  You  want  to  know  if  a  soldier  can  do  this  task. 

You  only  have  ten  minutes  for  the  test.  Which 
element  or  elements  would  you  want  to  see  him 
do?  •») 

7.  You  want  to  know  if  a  soldier  can  do  this  task. 

You  only  have  fifteen  minutes  for  the  test. 

Which  element  or  elements  would  you  want  to 
see  him  do?  * 

8.  You  want  to  know  if^a  soldier  can  do  this  task. 

You  only  have  twenty  minutes  for  the  test. 

Which  element  or  elements  would  you  want  to 
see  him  do? 

Two  questions  were  asked  to  determine  SME  opinion  of  the  most 
difficult  element.  The  two  questions  were: 

9.  Which  element  do^you  think  is  most  often 
performed  wrong? 

10.  Which  element  do  you  think  is  the  most  diffi- 
cult  to  do? 
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Two  questions  were  asked  to  determine  the  critical  elements. 

Tile  two  questions  were: 

11.  Sometimes  doing  a  task  involves  elements  that 
are  very  important  in  the  sense  that  doing 
them  wrong,  or  not  doing  them,  can  cause 
immediate  and  sometimes  irreversible  damage 
to  the  soldier  or  equipment  (pressing  the 
starter  button  on  a  tank  for  longer  than  15 
seconds).  Which  elements  in  this  task  can 
cause  immediate  damage  to  the  soldier  or  equip¬ 
ment  if  done  wrong  (or  not  done)? 

12.  Sometimes  doing  a  task  involves  elements  that 
are  very  important  in  the  sense  that  doing 

them  wrong,  or  not  doing  them,  can  be  potentially 
serious  for  the  soldier  or  equipment  but  not 
detected  while  the  task  is  being  done  (failing  . 
to  tighten  hub  nuts  on  a  1/4-ton  truck  to  the 
specified  torque).  Which  elements  in  this  task 
can  cause  damage  to  the  soldier  or  equipment 
if  done  wrong  (or  not  done)  but  might  not  be 
detected  while  the  task  is  being  done? 

Questionnai re^^Adrn inis  t r a t  ion 

The  questionnaire  was  administered  to  five  NCO  at  Ft.  Knox, 
Kentucky,  and  six  at  Aberdeen  Proving  Ground.  Table  4  describes  the 
SHE  by  PMOS  and  duty  position.  The  SME  worked  with  one  task  at  a 
time  and  answered  all  questions  for  the  task  before  starting  another 
task.  The  list  of  tasks  for  each  MOS  and  the  number  of  task  elements 
in  each  are  shown  in  Table  5.  The  task  elements  for  eacn  task 
reviewed  by  the  SHE  are  presented  at  Appendix  C.  It  is  important 
to  note  that  the  SME  only  responded  to  tasks  on  which  they  were 
familiar.  Therefore,  the  number  of  SME  responding  to  each  task  is 
not  always  the  same.  The  questions  were  given  orally  by  a  member 
of  the  project  staff;  the  SME,  however,  worked  independently 


RESULTS 


The  SME  selection  data  were  analyzed  for  inter-judge  agreement. 
In  order  for  the  selection  procedures  to  be  systematic  and  useful 
for  test  developers,  it  is  necessary  that  they  produce  consistent 
results.  High  agreement  among  SME  would  indicate  that  having  one  SME 
select  elements  would  be  as  valid  as  collecting  the  same  information 
from  any  other,  or  from  many  SME,  with  a  considerable  savings  in  time 
and  effort.  This  is  particularly  a  requirement  for  test  developers 
who  have  access  to  only  one  or  two  SME. 
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Table  4 


-MSL 


SME  By  PMOS  and  Duty  Position 


Location 

PMOS 

SME 

Number*  Duty  Position 

Ft.  Knox,  Kentucky 

63H40 

1 

Maintenance  Sergeant 

63H30 

2 

Material  Section 

63H30 

6 

Supply  Sergeant 

63B30 

5 

Maintenance  Sergeant 

63H20 

4 

Track  Vehicle  Inspector 

Aberdeen  Proving  Ground  , 

Maryland  63H40 

3 

Task  Analysis  Committee 

63H30 

11 

Task  Analysis  Committee 

63H30 

7 

SQT  Writer 

63H30 

8 

SQT  Writer 

63H30 

9 

Instructor,  Mobility  Branch 

63H30 

10 

Instructor,  Mobility  Branch 

*Randomly  assigned  for  analysis  purposes. 
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Table  3 


6311  and  63C  Tasks 


Task 


Number  of 
Task  Elements 


63H 

1.  Adjust  transmission  linkage  on  M113A1.  8 

2.  Inspect  M35A2  electrical  system.  12 

3.  Adjust  cam  dwell  on  M151A1/A2  truck.  12’ 

4.  Adjust  clutch  cover  assembly  on  M809  series 

truck.  7 

5.  Test  and  adjust  alternator  voltage  output  on 

M151A1/A2  truck.  6 

6.  Remove  and  replace  front  differential  on 

M151A1/A2  truck.  10 


63C 


1.  Troubleshoot  25  ampere  DC  charging  system.  8 

2.  Troubleshoot  starting  system  circuit  on 

M151A1/A2  truck.  11 

3.  Replace  steering  linkage  on  M151A1/A2  truck.  8 

4.  Troubleshoot  brakes  and  controls  on  M151A1/A2 

truck.  5 

5.  Troubleshoot  CD850  transmission,  M60A1.  12 

6.  Adjust  shift  control  linkage,  M60A1.  8 


■-at  ■ 


A  data  matrix  was  prepared  for  SME  responses  for  each  task.  The 
data  matrix  for  SME  responses  to  63H  task  #3  (Adjust  cam  dwell  on 
M151A1/A2  truck)  is  displayed  in  Figure  1.  The  data  matrixes  for  the 
remaining  tasks  are  at  Appendix  D.  Pairwise  agreements  among  SME 
were  then  computed  for  questions  1-4.  (For  the  convenience  of  the 
reader,  the  12  questions  are  listed  in  Table  6.)  On  question  1,  a 
pairwise  agreement  occurs  whenever  one  SME  selects  the  same  element 
as  another  SME.  If  four  SME  select  the  same  element,  six  pairwise 
agreements  are  counted.  Thus,  if  all  11  SME  respond  to  question  1 
on  a  given  task,  55  pairwise  agreements  are  possible.  On  question  2, 
two  types  of  agreement  can  occur:  two  SME  might  pick  the  same  two 
elements  (full  agreement),  or  they  might  agree  on  only  one  of  the  two 
elements  each  selected  (partial  agreement).  With  11  SME  responding, 
there  are  again  55  possible  pairwise  agreements,  but  110  possible 
partial  agreements.  On  question  3,  partial  agreement  is  possible  at 
two  levels  (agree  on  one  out  of  three  elements,  agree  on  two  out  of 
three  elements),  and  on  question  4  there  are  three  levels  of  partial 
agreement.  Partial  agreements  reported  in  Table  7  are  all  of  the 
"one-or-more-out-n"  sort.  The  figures  represent  obtained  agreement 
as  a  proportion  of  possible  agreement;  the  denominator  varies  as  a 
function  of  the  number  of  SME  responding  and  the  number  of  elements 
selected.  (Occasionally,  an  SME  would  select  fewer  than  the  number 
of  elements  asked  for.) 

Agreement  among  SME  was  disappointedly  low  on  all  questions. 

Full  agreement  on  the  first  four  questions  (Table  7)  ranged  over  tasks 
from  a  low  of  0  to  a  high  of  60%,  with  an  average  of  approximately 
13%.  Of  the  48  full  agreement  percentages  (four  questions  for  each 
of  12  tasks),  17  exceeded  chance  expectations.1  Statistical  signif¬ 
icance  is  not  the  only  criterion  or,  for  that  matter,  even  the  most 
important  criterion.  Practical  significance,  too,  must  be  considered. 
And  the  observed  levels  of  agreement  fall  substantially  below  that 
viewed  as  acceptable  for  practical  purposes. 


Chance  expectations  were  calculated  using  the  cumulative  binomial 
formula  (Lindgren,  B.W.  Statistical  Theory  (Second  Edition). 
Macmillan,  1968,  p.  150).  The  formula  gives  the  probability  of  any 
number  of  successes  in  a  number  of  trials,  given  the  probability  of 
success  on  a  single  trial.  Here,  a  success  is  defined  as  a  pair¬ 
wise  agreement  among  raters,  and  the  number  of  successes  ranges  from 
the  number  of  agreements  observed  to  the  number  of  trials  occurring. 

The  number  of  trials  is  the  number  of  pairs  of  raters.  The  probability 
of  success  (agreement)  on  a  single  trial  (pair  of  raters)  varies  as 
a  function  of  the  number  of  elements  in  the  task  and  the  number  of 
selections  the  rater  is  to  make.  On  question  1,  for  63H  Task  3, 
for  example,  there  are  12  elements,  and  the  nine  raters  are  each  to 
choose  one  element  for  testing.  The  number  of  trials  in  this  case 
is  36,  the  number  of  pairs  of  raters.  The  probability  of  agreement 
for  a  single  pair  of  raters  is  1/12.  Using  these  figures  in  the 
binomial  formula  for  11  to  36  agreements,  the  probability  of  obtaining 
11  or  more  agreements,  simply  by  chance,  is  .01.  This  may  be  compared 
to  the  6  or  more  agreements  of  36  which  would  be  expected  5%  of  the 
time. 
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k  Number  3  (63H) 

Dwell  on  M151A1/A2  Truck 


w 


Table  6 


Questions  for  SME  to  Select  Predictive  Element  j 


1.  You  want  to  know  if  a  soldier  can  do  this  task.  There  is  time  for 
him  to  do  only  one  of  the  elements  for  the  task.  Which  one  element 
would  you  want  to  see  him  do? 

2.  You  want  to  know  if  a  soldier  can  do  this  task.  There  is  time  for 
him  to  do  only  two  of  the  elements  for  the  task.  Which  two  elements 
would  you  want  to  see  him  do? 

3.  You  want  to  know  if  a  soldier  can  do  this  task.  There  is  time  for 
him  to  do  only  three  of  the  elements  for  the  task.  Which  three 
elements  would  you  want  to  see  him  do? 

4.  You  want  to  know  if  a  soldier  can  do  this  task.  There  is  time  for 
him  to  do  only  four  of  the  elements  for  the  task.  Which  f our  elements 
would  you  want  to  see  him  do? 

5.  You  want  to  know  if  a  soldier  can  do  this  task.  You  only  have  f ive 

minutes  for  the  test.  Which  element  or  elements  would  you  want  to 

see  him  do? 

6.  You  want  to  know  if  a  soldier  can  do  this  task.  You  only  have  ten 

minutes  for  the  test.  Which  element  or  elements  would  you  want  to 

see  him  do? 

7.  You  want  to  know  if  a  soldier  can  do  this  task.  You  only  have 

fifteen  minutes  for  the  test.  Which  element  or  elements  would  you 

want  to  see  him  do? 

8.  You  want  to  know  if  a  soldier  can  do  this  task.  You  only  have  twenty 

minutes  for  the  test.  Which  element  or  elements  would  you  want  to 

see  him  do? 

9.  Which  element  do  you  think  is  most  often  performed  wrong? 

0.  Which  element  do  you  think  is  the  most  difficult  to  do? 

1.  Sometimes  doing  a  task  involves  elements  that  are  very  important  in 
the  sense  that  doing  them  wrong,  or  not  doing  them,  can  cause  immediate 
and  sometimes  irreversible  damage  to  the  soldier  or  equipment  (pressing 
the  starter  button  on  a  tank  for  longer  than  15  seconds) .  Which 
elements  in  this  task  can  cause  immediate  damage  to  the  soldier  or 
equipment  if  done  wrong  (or  not  done)? 

2.  Sometimes  doing  a  task  involves  elements  that  are  very  important  in 
the  sense  that  doing  them  wrong,  or  not  doing  them,  can  be  potentially 
serious  for  the  soldier  or  equipment  but  not  detected  while  the  task 
is  being  done  (failing  to  tighten  hub  nuts  on  a  1/4-ton  truck  to  the 
specific  torque).  Which  elements  in  this  task  can  cause  damage  to  the 
soldier  or  equipment  if  done  wrong  (or  not  done)  but  might  not  be 
detected  while  the  task  is  being  done? 


Partial  agreement  results  for  questions  2-4  (Table  7)  though 
numerically  larger,  are  not  much  better  with  respect  to  chance 
expectations  than  were  results  for  full  agreement.  Responses  on 
questions  9-12  (Table  8)  show  similarly  low  SME  agreement,  the 
rates  typically  being  below  25%. 

Because  the  SME  responses  to  questions  5-8  (elements  to  test 
under  four  different  time  constraints)  varied  so  widely,  pairwise 
agreements  for  these  questions  were  not  computed.  However,  in  the 
interest  of  exploring  the  similarity  of  task  dimensions  being  con¬ 
sidered  by  raters  in  responding  to  these  questions  versus  questions 
1-4,  responses  to  the  two  sets  of  questions  were  correlated. 

Elements  on  each  task  were  rank-ordered  according  to  their  importance 
in  SME  selections.  An  element  selected  by  an  SME  in  response  to 
question  1  was  given  a  weight  of  4,  responses  to  question  2  were  given 
a  weight  of  3,  and  so  on.  Responses  to  questions  5-8  were  similarly 
weighted  and  rank-ordered.  Correlations  of  the  two  orders  (Table  9) 
ranged  widely — from  .10  to  .92  depending  on  the  particular  task 
involved.  No  evident  characteristics,  such  as  type  of  task  or  number 
of  elements,  however,  were  systematically  associated  with  these  dif¬ 
ferences  in  correlation. 


DISCUSSION 


Inter-judge  agreement  levels  were  so  low  that  no  attempt  was 
made  at  this  point  to  verify  the  selection  procedures  by  comparing 
the  ARTS  performance  data  with  the  SME  selected  elements.  Any 
discussion  of  the  causes  of  such  low  agreement  is,  of  course,  spec¬ 
ulative  in  nature  but  instructive  in  terms  of  modifying  the  selection 
instruments  and  procedures. 

One  would  anticipate  that  question  9  ("performed  wrong")  and 
question  10  ("most  difficult")  would  have  the  highest  agreement  not 
only  among  SME  but  also  within  an  SME.  Neither  of  these  expectations 
was  met  at  an  agreement  level  satisfactory  for  any  systematic  decisions. 
Agreement  among  SME  on  question  9  averaged  22.1%  on  the  63H  tasks  and 
23.7 %  on  the  63C  tasks;  agreement  on  question  10  averaged  39.3%  on 
the  63H  tasks  and  27.6%  on  the  63C  tasks.  The  within  SME  agreement 
was  40%  across  all  the  tasks;  that  is,  only  40%  of  the  time  did  an 
SME  select  the  same  elements  for  both  questions.  The  low  agreement 
among  SME  is  probably  a  result  of  the  experience  factor;  whatever 
element  is  a  problem  for  the  SME  when  he  performs  the  task  is  the 
element  he  selects.  The  low  agreement  within  an  SME  is  most  likely 
a  result  of  different  interpretations  of  the  two  questions. 

Questions  1  through  4  seem  reasonable  from  an  element  selection 
view  point  but  agreement  was  much  too  low.  Agreement  increases  with 
increases  in  the  number  of  elements  one  is  permitted  to  select; 
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Table  8 


Pairwise  Agreement  Among  SME 
On  Selection  Questions  9-12 


_ Question 

Number  of 


Task 

Elements 

9 

10 

11 

12 

63H  1 

8 

4/10 

3/10 

3/19 

2/10 

2 

12 

8/53 

28/55 

22/87 

14/127 

3 

12 

2/36 

18/36 

9/36 

10/28 

4 

7 

10/52 

15/35 

4/21 

12/44 

3 

6 

16/36 

16/36 

7/28 

4/35 

6 

10 

4/45 

8/45 

15/54 

19/85 

AVERAGE 

22.1% 

39.3% 

23.0% 

21.3% 

63C  1 

8 

2/20 

3/15 

5/26 

4/20 

2 

11 

4/32 

3/33 

5/24 

3/14 

3 

8 

19/86 

8/53 

10/53 

26/98 

4 

5 

36/45 

11/36 

4/28 

23/94 

5 

12 

2/14 

6/10 

1/9 

1/6 

6 

8 

3/9 

4/13 

0/3 

0/6 

AVERAGE 

23.7% 

27.6% 

14.0% 

18.2% 

*See  Figure  1  for  raw  data. 


Tab. U-  9 


Correlation  in  Rank  Order  of  Elements 
As  Assigned  by  SME  on  Selection  Questions  1-4  and  Questions  5-8 


Number  of 


Task 

Elements 

Rho 

Si gn i f i cance 

63H  1 

8 

.5952 

NS 

2 

12 

.4021 

NS 

3 

12 

.  3077 

* 

NS 

4 

7 

.8571 

A 

O 

Ln 

5 

6 

.8286 

A 

• 

o 

6 

10 

.2242 

NS 

63C  1 

8 

.  7738 

p  ■  .05 

2 

11 

.4955 

NS 

3 

8 

.7619 

r>  *  .05 

4 

5 

.1000 

NS 

5 

12 

?*o 

00 

NS 

6 

8 

.9226 

p  <  .05 

*See  Figure  1  for  raw  data. 
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however,  that  is  to  be  expected  since  the  opportunity  for  agreement 
is  increased.  The  low  agreement  among  SME  for  the  first  question, 
averaging  20.7%  for  the  63H  tasks  and  25.9%  for  the  63C  tasks,  was 
discouraging  since  this  question  seemed  the  more  simple  and  unambig¬ 
uous  of  those  asked. 

Little  more  can  be  said  regarding  SME  agreement  levels.  The 
selection  instrument  could  be  modified  by  deleting  the  questions 
concerning  time  limits  (questions  5  through  8)  as  these  seemed  the 
most  ambiguous  to  SME;  questions  3  and  4,  regarding  additional 
elements  to  test,  could  also  be  deleted  since  SME  felt  that  two 
elements  were  sufficient.  But  these  deletions  would  still  leave 
questions  which,  without  more  experienced  SME  or  perhaps  SME  better 
trained  in  making  such  analytic  judgments,  cannot  be  answered  reliably. 
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CHAPTER  4 

STUDY  III:  IDENTIFYING  PREDICTIVE  ELEMENTS  EMPIRICALLY 


Since  the  selection  of  predictive  elements  by  SME  proved  unreliable 
and  therefore  unusable,  another  approach  was  tried.  The  performance 
data  for  the  ARTS  tasks  were  examined  to  determine  empirically  the 
most  likely  testing  point(s)  within  each  task.  If  key  testing  points 
(i.e.,  elements  highly  predictive  of  total  task  performance)  could  be 
identified  in  this  way,  it  might  then  be  possible  to  develop  from 
the  empirical  data  procedures  for  test  developers  that  would  enable 
them  to  select  these  same  testing  points  without  benefit  of  test  results 
For  example,  there  may  be  a  type  of  element  among  all  adjustment  tasks 
(63H  tasks  1,  3,  and  4;  63C  task  6)  that  is  typically  performed  incor¬ 
rectly  by  nonperformers,  and  typically  performed  correctly  by  performers 
that  type  of  element  could  then  be  identified  as  a  key  testing  point 
in  test  development  procedures  for  that  type  of  task. 


METHOD 


The  objective  was  to  identify  the  most  predictive  task  element (s) 
and  the  most  difficult  task  element(s)  for  each  of  the  twelve  tasks, 
examine  these  elements  for  any  characteristics  which  tie  them  together, 
and  then  develop  rules  for  selecting  elements  with  these  same  charac¬ 
teristics.  First,  any  soldier  who  did  not  complete  the  task  was 
eliminated  from  the  data  base  for  that  task.  Task  completion  is  not 
to  be  confused  with  successful  task  performance;  soldiers  were  per¬ 
mitted,  in  fact,  encouraged,  to  continue  the  task  if  they  made  an 
error  on  one  or  more  tusk  elements.  The  number  of  soldiers  completing 
each  63H  task  and  each  63C  task  and  used  in  the  analyses  is  presented 
in  Tables  10  and  11,  respectively. 

When  the  tasks  were  administered  for  record  (during  the  ARTS) , 
scorers  were  permitted  to  "prompt"  soldiers  who  were  having  difficulty 
performing  a  task  element;  if  he  then  performed  the  element  success¬ 
fully,  the  soldier  was  given  a  GO  for  the  element.  The  scorer,  however, 
indicated  on  the  scoresheet  the  number  of  prompts,  if  any,  given  for 
the  element.  Any  task  element  that  had  a  GO  with  a  prompt  was  con¬ 
verted  to  a  NO  GO.  This  was  done  for  two  reasons.  First,  if  a  soldier 
needs  a  prompt,  he  doesn't  know  how  to  perform  the  task  element  and 
second,  the  number  of  NO  GO  was  increased  to  give  a  more  acceptable 
variance  in  scores. 

Once  the  data  were  "clean" — that  is,  soldiers  eliminated  who 
did  not  complete  the  task — and  a  GO-with-a-prompt  for  any  task  element 
converted  to  a  NO  GO,  two  statistical  analyses  were  performed: 

Forward  Stepwise  Multiple  Regression,  and  Guttman  Scalogram  Analysis. 
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Table  10 


63H  Tasks 


Number  of 


Task  Soldiers3 

1.  Adjust  transmission  linkage  on  M113A1.  151 

2.  Inspect  M35A2  electrical  system.  108 

3.  Adjust  cam  dwell  on  M151A1/A2  truck.  90 

4.  Adjust  clutch  cover  assembly  on  M809  series  truck.  79 

5.  Test  and  adjust  alternator  voltage  output  on 

M151A1/A2  truck.  105 

6.  Remove  and  replace  front  differential  on  M151A1/A2 

truck.  132 


This  is  the  number  of  soldiers  who  completed  the  task  out  of  the  190 
who  began  the  task. 
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^2? 


Table  11 
63<"  Tasks 


Task 


Number  of 
Soldiers1 


1.  Troubleshoot  25  ampere  DC  charging  system. 

2.  Troubleshoot  starting  system  circuit  on  M151A1/A2 
truck . 

3.  Replace  steering  linkage  on  M151A1/A2  truck. 

4.  Troubleshoot  brakes  and  controls  on  M151A1/A2  truck. 

5.  Troubleshoot  CD850  transmission,  M60A1. 

6.  Adjust  shift  control  linkage,  M60A1 . 


101 

96 

135 

131 

128 


This  is  the  number  of  soldiers  who  completed  the  task  out  of  the  137 
who  began  the  task. 
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1 


For ward  Stepwise  Multiple  Re g re s s i on 

This  is  a  procedure  through  which  one  can  analyze  the  relationship 
between  a  dependent  or  criterion  variable  and  a  set  of  iniependent  or 
predictor  variables.1  For  the  ARTS  data,  total  test  score  is  the 
criterion  and  each  task  element  is  a  potential  predictor.  In  pred i r t inf 
values  of  the  criterion  or  dependent  variable  from  the  set  of  predii  tor 
or  independent  variables,  the  regression  analysis  first  selects  the 
independent  variable  which  has  the  highest  correlation  with  the  depen¬ 
dent  variable.  This  correlation,  when  squared,  expresses  the  variance 
in  the  dependent  variable  which  is  accounted  for  by,  or  predictable 
from,  the  independent  variable.  That  independent  variable's  influence 
is  then  partialled  out;  the  next  independent  variable  to  enter  the 
regression  is  the  one  which  accounts  for  the  greatest  proportion  of 
variance  in  the  dependent  variable  which  is  not  yet  accounted  for  by 
the  leading  independent  variable.  The  process  continues,  at  each  step 
producing  a  (multiple)  correlation  which,  squared,  indicates  the 
variance  accounted  for  by  all  independent  variables  entered  up  to  that 
step.  The  standard  error  of  estimate  computed  at  each  step  indicates 
the  average  error  in  prediction  which  would  occur  if  the  dependent 
variable  was  predicted  from  the  independent  variables  entered  up  to 
that  step. 


Guttman  Scalogram  Analysis 

This  procedure  analyzes  the  underlying  operating  characteristics 
of  items  to  determine  if  their  interrelationships  meet  several  special 
properties  which  define  a  Guttman  scale.  First,  the  items  must  be 
unidimensional;  that  is,  they  must  all  measure  a  single  underlying 
object  or  ability.  Second,  the  items  must  be  cumulative;  that  is, 
they  can  be  ordered  by  degree  of  difficulty  as  indicated  by  respondents 
who  pass  a  difficult  item  always  passing  less  difficult  items,  and 
vice  versa.2  The  elements  are  thus  arranged  in  order  of  difficulty, 
and  certain  statistical  tests  applied  to  determine  how  well  the  data 
actually  conform  to  these  expected  tendencies.  These  tests  measure 
whether  elements  are  unidimensional  and  cumulative;  that  is,  whether 
they  tap  the  same  underlying  abilities  and  are  more  or  less  hierarchial 
in  their  requirement  for  demonstration  of  the  underlying  ability. 

The  tests  are:  (a)  Coefficient  of  Reproducibility  (CR) ,  which  is  a 
measure  of  the  extent  to  which  a  person's  total  task  score  predicts 
his  task  element  pass  pattern;  (b)  Minimum  Marginal  Reproducibility 
(MMR) ,  which  gives  the  minimum  CR  that  could  have  occurred  for  the 
task  given  the  proportion  of  people  passing  and  failing  each  element; 


^ie,  N.H.,  Hull,  C.H.,  Jenkins,  J.G.,  Steinbrenner ,  K. ,  and  Bent,  D.H. 
SPSS:  Statistical  Package  for  the  Social  Sciences  (Second  Edition). 
McGraw-Hill,  1975. 

2Ibid. 
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(c)  Percent  Improvement  (PI),  or  the  difference  between  CR  and  MMR ; 
and  (d)  Coefficient  of  Scalability  (CS) ,  which  provides  an  indication 
of  the  extent  to  which  the  scale  is  truly  unidimensional  and  cumulative. 


RESULTS 


The  ARTS  performance  test  results  were  examined  and  two  hypotheses 
tested  by  means  of  the  multiple  regression  and  scalogram  analyses. 

The  two  hypotheses  were: 

1.  The  regression  analysis  woulf  identify  a  subset 
of  elements  of  each  task  which  would  predict 
task  performance;  that  is,  would  account  for 
more  of  the  variance  in  total  task  scores. 

2.  The  Guttman  scale  analysis  would  provide  a 
reliable  ordering  of  the  elements  of  each  task; 
that  is,  the  elements  would  prove  to  be  uni¬ 
dimensional  and  cumulative. 


Regression  Analysis 

In  the  multiple  regression  analysis  the  ARTS  tasks  were  analyzed 
individually.  The  independent  variables  were  outcomes  on  each  task 
element  (pass  or  fail),  and  the  dependent  variable  was  total  task  score, 
the  number  of  elements  passed.  Thus,  unlike  most  regression  problems, 
it  is  always  possible  to  predict  the  dependent  variable  without  error, 
simply  by  adding  the  values  of  all  the  independent  variables.  The 
purpose  of  the  multiple  regression  analysis  was  to  determine  the  sub¬ 
set  of  independent  variables  (elements)  which  could  together  account 
for  most  of  the  variance  in  the  dependent  variable  (task  performance) , 
and  predict  the  dependent  variable  within  acceptable  error  limits. 

The  means  and  standard  deviations  for  the  tasks  are  presented  in 
Appendix  E;  the  regression  summary  scores  for  the  tasks  are  in  Appendix  F. 

As  may  be  seen  in  Table  12,  at  least  half  of  the  task  elements, 
in  all  but  two  tasks,  had  to  be  entered  into  the  regression  in  order 
to  account  for  90%  of  the  variance  in  total  score.  In  five  of  the 
tasks,  more  than  half  of  the  elements  each  accounted  for  up  to  5%  of 
the  variance. 

In  these  data,  it  is  apparent  that  any  attempts  to  determine  task 
performance  from  knowledge  of  performance  on  only  a  subset  of  elements 
will  result  either  in  considerable  prediction  error,  or  in  miniscule 
savings  in  testing  time.  But  a  closer  examination  of  the  data  reveals 
possible  mitigating  circumstances  which  may  explain  why  so  many  elements 
had  to  enter  the  regressions  for  relatively  error-free  prediction. 


Table  12 


Summary  of  Multiple  Regression  Analysis  Where 
Dependent  Variable  Is  Task  Score  (Elements  Passed) 
And  Independent  Variables  Are  Task  Elements 


Task 

In 

Task 

Number  of  Elements: 

Required  To 

Account  For 

90%  of 

Variance 

Contributing 

At  Least  5% 

To  Variance 
Accounted  For 

63H  1 

8 

4 

3 

2 

7a 

6 

7 

3 

13a 

5 

3 

4 

7 

4 

4 

5 

7a 

5 

5 

6 

10 

5 

4 

63C  1 

8 

6 

7 

2 

11 

7 

4 

3 

8 

3 

3 

4 

5 

4 

5 

5 

12 

8 

5 

6 

8 

5 

4 

aThe  number  of  task  elements  on  which  examinees  were  scored  is  different 
from  the  number  reviewed  by  SME  because  of  task  analysis  modification 
(discussed  earlier) . 
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Although  the  data  were  initially  adjusted  so  that  prompted 
elements  were  considered  as  failures  to  perform,  the  variance  within 
tasks  was  still  small,  and  the  distribution  of  scores  was  markedly 
skewed  (see  Table  13).  The  ideal  data  for  producing  a  promising 
regression  would  have  total  scores  distributed  somewhat  normally, 
with  an  average  of  around  half  the  elements  passed  and  frequency  of 
scores  trailing  off  toward  the  t_. Cremes  of  perfect  performance  or 
complete  failure.  Element  difficulties  would  also  spread  normally, 
between  .8  and  .2  (ideally,  all  would  be  close  to  .5).  Element 
correlations  would  be  somewhat  clustered;  that  is,  some  elements 
would  have  high  intercorrelations  with  each  other,  and  low  correlations 
with  other  groups  of  elements,  each  of  which  would  also  correlate 
highly  among  themselves.  There  would  be  few,  if  any,  negative  cor¬ 
relations,  and  no  high  negative  correlations.  Correlations  between 
elements  and  total  score  would  all  be  moderate,  with  at  least  one 
element  in  each  group  of  highly  correlated  elements  having  a  relatively 
high  correlation  with  total  score. 

In  the  ARTS  data,  these  conditions  did  not  obtain.  Because  of 
the  way  in  which  performance  data  are  necessarily  collected  and  the 
sequential  nature  of  task  elements,  a  failure  on  a  task  element 
often  means  that  the  examinee  cannot  even  attempt  subsequent  elements. 
Regression  analysis  of  the  type  performed  requires  complete  data  on 
all  subjects;  data  for  those  who  did  not  complete  the  task  could  not 
be  included  in  the  analysis.  The  result  is  that  element  difficulties 
(percent  failing)  are  spuriously  low.  Even  though  examinees  tended 
to  fail  seemingly  random  elements  (an  examinee  with  a  high  score  did 
not  necessarily  fail  only  the  more  difficult  elements,  nor  did  an 
examinee  with  a  low  score  pass  only  the  easier  elements) ,  the  large 
numbers  of  examinees  passing  each  element  produced  many  moderate 
correlations  among  elements  and  no  clear  clusters  of  elements.  Many 
also  tended  to  be  highly  correlated  with  total  scores.  Together, 
this  meant  that  the  first  element  in  the  regression  might  account  for 
a  sizable  chunk  of  the  available  variance,  but  subsequent  elements 
were  able  to  contribute  little  to  the  prediction  beyond  what  was 
already  known. 

If  the  regression  were  to  be  done  with  a  simple  GO/NO  GO  on  task 
performance  as  the  dependent  variable,  the  order  of  entry  of  the  task 
elements  into  the  regression  would  change.  The  elements  would  then 
enter  in  order  of  difficulty.  This  has  intuitive  appeal:  the  more 
difficult  elements,  if  performed  correctly,  should  indicate  something 
about  ability  to  perform  the  less  difficult  elements,  and,  by 
extension,  about  ability  to  perform  all  elements  correctly.  Although 
the  predictive  order  of  the  elements  would  be  reasonable,  the  variance 
accounted  for  would  be  small,  and  the  prediction  fraught  with  error. 


Table  13 


element  difficulty  is  computed  as  the  proportion  of  examinees  failing  each  element, 
across  elements  in  the  task. 


Guttman  Scalogram  Analysis 


Of  the  12  tasks  examined,  none  formed  a  scale  (Table  14).  Though 
three  tasks  had  CR  values  greater  than  .9,  the  criterion  for  a  valid 
scale,  none  had  a  PI  index  that  exceeded  8%,  indicating  that  examinees' 
total  scores  reveal  little  or  nothing  about  which  items  they  passed 
or  failed.  In  other  words,  a  low  test  score  does  not  necessarily 
mean  that  the  examinee  failed  all  of  the  more  difficult  items  nor 
passed  only  the  easiest  items;  nor  does  a  high  score  indicate  that 
the  examinee  failed  only  the  most  difficult  items.  Any  high  CR  in 
this  case  is  due  more  to  inherent  cumulative  interrelationships  of 
the  elements  than  to  performance  patterns.  The  low  values  of  CS, 
which  should  be  well  above  .6  if  the  scale  is  truly  unidimensional 
and  cumulative,  indicate  that  the  tests  of  these  tasks  are  probably 
not  unidimensional,  and  certainly  not  cumulative. 

To  determine  whether  the  tasks,  in  fact,  are  not  unidimensional, 
internal  consistency  estimates  were  computed  for  each  task  using 
Kuder-Richardson  Formula  20.*  As  shown  in  Table  14,  only  two  tasks 
had  what  might  be  considered  high  internal  consistency  (r  =  .80). 

The  rest  were  low  to  moderate,  indicating  that  there  is  no  common  set 
of  abilities  uniformly  required  by  all  the  elements.  More  likely, 
all  elements  in  a  task  are  linked  by  virtue  of  having  been  learned  at 
the  same  time,  but  may  have  been  learned  to  varying  degrees  of  compe¬ 
tency.  In  addition,  the  elements  probably  do  require  different  skills 
and  abilities  since  maintenance  tasks  tend  to  be  heterogeneous,  and 
one  would  expect  low  consistency.  If  it  were  possible  to  separate 
knowledge  of  task  procedures  (knowing  what  to  do,  where,  and  when) 
from  ability  to  perform  those  procedures  (the  basic  skills  or  abilities), 
the  underlying  skills  might  appear  more  str  ngly.  Each  task  would  be 
more  heterogeneous  as  a  whole,  but  would  be  composed  of  subsets  of 
homogeneous  elements. 


Empirical  Versus  SME  Selections 


Despite  the  low  reliability  of  SME  selections  in  the  previous  study, 
and  at  the  risk  of  compounding  measurement  error,  it  was  decided  to  check 
SME  judgments  against  results  of  the  empirical  analysis.  Since  agree¬ 
ment  among  SME  was  so  low,  further  analyses  of  the  SME  selections  were 
confined  to  comparisons  of  the  various  methods  of  selection  to  see  if 
any  were  more  likely  to  elicit  responses  which  agreed  with  the  results 
of  the  ARTS  data  analyses.  Questions  1-4  were  treated  as  four  separate 
methods  of  task  element  selection.  For  question  1,  an  agreement  was 
counted  whenever  an  SME  selected  the  element  which  was  the  lead  predictor 
in  the  regression  analysis.  For  question  2,  a  full  agreement  occurred  when 


*Guilford,  J.P.  Psychometric  Methods 
Hill,  1954,  p.  380. 


(Second  Edition) .  McGraw- 


Table  14 

Guttman  Scale  Analysis  and 
Internal  Consistency 


Guttman  Scale 

Test  Statistics 

a 

Test 

CR 

MMR 

n 

cs 

Internal 

Consistency 

63H  1 

.926 

.890 

3.6% 

.323 

.592 

2 

.783 

.728 

5.6% 

.204 

.524 

3 

.837 

.822 

1.5% 

.083 

.84  3 

4 

.879 

.846 

5.2% 

.341 

539 

5 

.815 

.736 

7.9% 

.299 

.482 

6 

.980 

.983 

1 

OJ 

1 

.182 

.568 

63C  1 

.774 

.718 

5.6% 

.198 

.217 

2 

.811 

.798 

1.3% 

.062 

.579 

3 

.974 

.958 

1.6% 

.375 

.811 

4 

.787 

.713 

7.4% 

.258 

.442 

5 

.838 

.836 

.2% 

.015 

.601 

6 

.846 

.807 

3.9% 

.202 

.652 

3 

See  text. 
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the  SME  selected  the  two  lead  elements  ,  and  a  partial  agreement  when  one 
of  the  two  lead  elements  was  selected.  With  questions  3  and  4,  there 
are  two  and  tnree  levels  of  partial  agreement,  respectively.  The 
agreements  were  added  across  tasks,1  for  full  and  partial  agreements 
on  each  question,  for  each  SME.  The  sums  were  then  converted  to 
proportions  (agreements  divided  by  number  of  selections)  and  averaged 
across  SME.  Similarly,  the  expected  agreements  were  calculated  for 
all  full  and  partial  agreements  for  each  SME  and  averaged  across  SME. 

The  results  are  shown  in  Table  15,  with  expected  agreements  in 
parentheses  next  to  each  observed  agreement.  A  goodness-of-f it  test2 
indicated  that  questions  2  and  3  elicited  more  full  agreements  than 
would  be  expected  by  chance.  Such  results,  however,  are  not  strong 
enough  to  decide  on  a  preferred  method  of  element  selection. 

Question  9,  concerning  the  element  most  often  performed  wrong, 
was  compared  with  the  obtained  difficulty  levels  of  elements.  Across 
tasks,  SME  selected  the  most  difficult  element  in  16.1%  of  their 
selections.  When  an  agreement  was  counted  if  they  selected  either 
of  the  two  most  difficult  items,  their  agreement  rate  was  35.5%; 
when  the  standard  was  further  relaxed  to  any  of  the  three  most  dif¬ 
ficult  items  in  a  task,  their  rate  was  still  only  45.3%.  None  of 
these  rates  is  acceptable  in  terms  of  a  systematic  selection  method. 


DISCUSSION 


A  statistical  approach  to  task  element  analysis  of  the  sort 
attempted  here  is  clearly  appropriate  and  feasible.  In  this  partic¬ 
ular  case,  however,  scoring  shortcomings  and  variance  restrictions 
in  the  data  tended  to  limit  opportunities  for  meaningful  results. 

Too  many  task  elements  tended  to  emerge  in  achieving  predictability 
of  total  task  score.  Moreover,  the  elements  that  were  identified  as 
most-predictive  seemed  to  have  little  in  common,  whether  viewed 
within  a  task,  a  task  category,  or  over  all  tasks. 

Inconclusive  though  these  results  were,  it  is  difficult  to 
relinquish  the  belief  that  "most-predictive"  elements,  if  reliably 
established,3  indeed  have  something  in  common.  It  seems  only  reasonable 
that  within  a  set  of  highly  similar  tasks  whatever  underlies  the 
predictiveness  of  one  most-predictive  element  also  underlies  the  pre- 
dictivensss  of  other  most-predictive  ones.  The  problem  is  to  capture 
that  underlying  construct.  And  to  do  this  one  needs  to  know  not  only 


3Data  for  the  three  tasks  for  which  task  analyses  were  modified  (63H, 
tasks  2,  3,  and  5)  were  not  included  in  this  analysis. 

2Lindgren,  B.W.  Statistical  Theory  (Second  Edition),  p.  325. 

3It  should  be  noted  that  in  studies  such  as  this  the  reliability  of 
element  predictiveness  should  be  verified  through  cross-validation 
on  a  second  sample  of  examinees. 
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Table  15 


Observed  and  Expected  Agreements 
Between  SME  Selections  and  Empirical  Selections 


Question 


Agreement 

r 

1 

2 

3 

4 

No  Agreement 

76.5 

(86.4)* 

45.0 

(50.4) 

16.8 

(16.9) 

3.5 

(3.1) 

One  Element 

23.5 

(13.6) 

45.5 

(44.9) 

35.6 

(47.5) 

20.0 

(21.0) 

Two  Elements 

- 

9.3 

(4.7)** 

41.5 

(32.2) 

26.5 

(36.6) 

Three  Elements 

- 

- 

5.5 

(3.4)* 

43.6 

(33.7) 

Four  Elements 

- 

- 

- 

6.4 

(5.5) 

At  Least  One  Element 

- 

32.2 

(27.2) 

45.3 

(40.6) 

55.2 

(54.4) 

*p  <  .10 

**p  <  .05 
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what  task  elements  an  examinee  misperformed  but  exactly  why  or 
precisely  what  aspect  of  the  element  was  failed.  The  importance 
of  such  detailed  diagnostic  data  is  illustrated  in  the  following 
example.  Consider  two  maintenance  tasks  each  with  an  element  t hat 
requires  torqueing  a  bolt,  and  each  with  an  element  that  requires 
testing  the  vacuum  in  a  line.  Further,  suppose  that  the  predominant 
correlate  of  failure  on  Task  A  was  failure  on  the  bolt-torqueing 
element,  whereas  on  Task  B  it  was  the  vacuum  test.  One  would  be 
tempted  to  attribute  such  results  to  a  lack  of  internal  consistency 
over  the  tasks.  That  is,  why  would  poor  performers  fail  bolt-torqueing 
(or  vacuum  testing)  on  one  task  but  pass  it  on  another?  If  there  is 
an  underlying  common  skill,  what  is  it?  Further  probing  might 
reveal  that  these  "most-predictive"  elements  both  involved  system 
components  that  were  inaccessible,  that  the  modal  reason  for  failure 
had  nothing  to  do  with  how  to  use  a  torque  wrench  or  vacuum  gauge, 
but  with  how  to  locate  equipment  points  that  are  inaccessible  or 
difficult  to  identify  from  the  job  aid.  Thus,  an  underlying  skill, 
different  in  kind  from  that  expected,  may  be  found  which  behaviorally 
ties  together  statistically  reliable  but  ostensibly  unrelated  pass- 
fail  patterns  over  tasks.  The  ARTS  data  available  in  this  study 
unfortunately  did  not  offer  the  detailed  diagnostic  information 
needed  to  carry  out  this  sort  of  penetrating  analysis. 

It  should  be  noted,  however,  that  even  if  a  thorough  analysis 
of  diagnostic  data  was  done  and  the  underlying  source(s)  of  commonality 
located,  the  result  may  be  of  little  use  in  test  development.  Most- 
predictive  elements  may  be  mediated  by  factors  that  are  difficult,  if 
not  impossible,  to  generalize  to  new  tasks  in  the  course  of  fore¬ 
casting  their  relevance  for  test  development.  Most-predictive  elements 
may  be  united  by  such  underlying  factors  as  elements  poorly  covered 
in  training,  or  elements  that  cannot  be  observed,  or  those  poorly 
described  in  the  job  aid. 
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CHAPTER  5 


GENERAL  DISCUSSION 


If  time-  and  resource-consuming  tasks  are  to  be  tested  feasibly, 
a  way  of  testing  just  the  more  relevant  task  behaviors  must  be  found. 
Any  such  method  must  entail  test  developers  forecasting  reliably 
those  task  elements  most  predictive  of  whole  task  performance.  The 
forecasting,  moreover,  cannot  proceed  task  by  task  from  an  empirical 
analysis  of  element  task  correlations — that  simply  is  not  practical. 
Needed  are  guidelines  which,  by  type  of  task  and  configuration  of  task 
elements,  enable  the  developer  to  choose  reliably  the  most  predictive 
subset  of  elements  to  test. 

Results  of  the  work  reported  here  indicate  that  task  experts 
are  not  able  to  select  relevant  elements  for  testing,  at  least  for 
the  sample  of  maintenance  tasks  used.  Since  experts  were  not. able 
to  agree  in  their  selections  of  relevant  behaviors  to  sample  for 
testing,  it  is  evident  that  subjective  approaches  to  identifying  pre¬ 
dictive  elements  of  tasks  for  testing  purposes  are  not  fruitful. 

More  must  be  learned,  first,  about  the  nature  of  causes  underlying 
the  internal  consistency  (or  inconsistency)  of  task  performance  and. 
second,  about  how  these  causes  change  systematically  from  one  class 
of  tasks  to  another.  Only  then  can  the  test  developer  be  given  useful 
guidance  for  sampling  task  performance. 

Empirical  analyses  in  the  general  form  of  Study  III  reported  here 
should  be  carried  out.  But  they  must  be  more  carefully  designed. 

At  a  minimum,  tney  should  meet  three  requirements: 

1.  Enough  examinees,  with  a  range  of  ability  on 
the  tasks,  must  be  tested  to  insure  variance 
in  performance  scores. 

2.  The  performance  data  must  include  not  only 
the  pass/fail  scores  on  elements,  but  also 
the  reason  why  an  examinee  fails  an  element. 

3.  The  tasks  tested  must  represent  all  skills 
required  for  the  job. 

The  first  requirement,  that  there  be  many  examinees  and  that  the 
data  have  a  high  variance,  is  important  in  order  to  draw  valid  and 
reliable  conclusions  concerning  predictive  elements.  If  there  are 
many  examinees  who  all  do  very  well  or  very  poorly  on  the  test  of  a 
task,  no  inferences  are  possible  concerning  which  elements  are  pre¬ 
dictive  of  ability  to  perform.  If  performance  varies  widely  but  only 
a  few  examinees  are  tested,  any  conclusions  would  be  suspect  by  reason 


of  statistical  error.  The  most  inst-rf  feel  ivi  method  i  ■  t  tost  in;1  r,.iir. 
examinees  is  to  focus  on  sliort  tasks,  with  low  to  nmiii  r.iti  time  am' 
equipment  requirements.  In  order  to  insure  control  over  t lie  vuri  am  o 
in  performance,  a  more  ambitious  approacii  would  involve  training 
examinees  to  varying  levels  of  competency.* 

The  second  requirement,  that  performance  data  he  more  diai'im:  - 
tically  precise,  makes  possible  a  more  definitive  statement  of  reasons 
for  performance  deficiency.  Some  elements,  although  stated  concise  I ;, 
will  involve  more  than  one  behavior  even  though  the  observable, 
scorable  behavior  is  unitary.  For  example,  the  element,  "(onnv.it 
transmission  vent  line,"  is  easilv  observed  and  scored.  however,  i! 
the  examinee  receives  a  fail  for  the  element,  we  do  not  know  whether 
he  simply  didn't  do  it,  or  didn't  know  where  or  how  to  connect  tiu 
vent  line,  or  didn't  know  what  the  transmission  vent  lint  was,  or 
didn't  connect  it  properly.  Two  elements,  seemingly  unrelated  in 
behavioral  requirements,  may  cluster  as  predictors  because  of  common 
underlying  reasons  for  failure.  Each  element  must  he  broken  down 
into  underlying  skill  requirements. 

The  third  requirement,  that  the  tasks  selected  for  testing  represent 
all  skills  required  for  the  job,  insures  that  the  elements  which  art- 
determined  to  be  predictive  of  performance  are  in  fact  the  most  pre¬ 
dictive.  With  all  skills  represented,  it  would  be  possible  to  determine 
clusters  of  elements  which  are  predictive  and  generalizable  across 
tasks,  rather  than  simply  within  tasks.  The  skills  to  be  represented 
should  be  non-task-spec  1 f ic ,  such  as  those  defined  by  Powers*  (Figure  2 )  . 

Data  which  meet  these  three  requirements  may  then  be  used  to 
determine  which  elements,  or  specifically,  which  skill  requirements 
are  predictive  of  job  proficiency.  It  may  be  that  a  different  skill 
taxonomy  is  required.  It  may  even  be  that  no  useful  skill  taxonomv 
exists  for  maintenance  tasks,  or  that  elements  which  fall  no.it  lv  into 
skill  categories  are  still  independent  in  terms  of  predicting  pro¬ 
ficiency.  But  if  useful  skill  categories  are  defined,  and  if  task 
elements  may  be  reliably  assigned  to  categories  on  a  rational  basis 
(as  opposed  to  empirical  assignment),  then  an  efficient  method  of 
testing  may  be  developed. 

For  example,  consider  the  display  shown  in  Figure  2.  If  tin- 
columns  represent  verified  generalizable  skills,  and  tlu  x  within 
column  indicates  that  one  or  more  elements  of  a  task  require  that 
skill,  two  approaches  would  be  warranted.  First,  the  test  developer 
might  administer  a  collection  of  skill  tests,  not  necessarily  part 


^Osborn,  W.C.  and  Ford,  J.P.  Research  on  Methods  of  Synthetic  Per- 
formance  Testing.  HumRRO  Final  Report  FR-CD(L) -76-1 ,  Apr i 1  1976. 

^Powers,  T.E.  Selecting  Presentation  Modes  Accord i ng  to  Personnel 
Characteristics  and  the  Nature  of  Job  Tasks.  Part  1:  Job  Tasks. 
University  of  Maryland  Baltimore  County,  January  1977. 
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of  any  task,  so  that  each  skill  is  demonstrated  at  the  end  of  the  course. 
Another  approach  might  be  to  select  tasks  for  testing  in  such  a  way 
that  each  skill  is  demonstrated.  The  skill  test  approach  has  the 
advantage  of  selecting  the  "most  generalizable"  element  for  testing 
(if  such  a  thing  is  found  to  exist).  In  that  way,  the  approach  reduces 
sampling  error.  The  comprehensive  task  approach  has  the  advantage 
of  yielding  direct  information  about  ability  to  perform  some  of  the 
domain.  Both  approaches  offer  the  opportunity  for  inferring  pro¬ 
ficiency  on  untested  behaviors. 
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APPENDIXES 


APPENDIX  A 

TASK  ANALYSIS  REVIEW  AND  MODIFICATION  PROCEDURE 


TASK  ANALYSIS  REVIEW  AND  MODIFICATION  PROCEDURE 


Item  1 


Some  maintenance  activities  take  a  long  time  to  complete. 
For  example,  it  takes  six  to  eight  tiours  to  replace  the 
cannon  tube  and  four  to  six  hours  to  replace  the  counter 
recoil  buffer  on  the  M109A1  SP  Howitzer.  There  are, 
however,  some  natural  breaks  or  separations  in  most  of 
the  activities.  For  purposes  of  the  task  analysis  review, 
we  will  refer  to  these  breaks  as  subtasks.  Replace  the 
counter  recoil  buffer,  then,  breaks  into  six  subtasks: 


Subtask  1 
Subtask  2 
Subtask  3 
Subtask  A 
Subtask  5 
Subtask  6 


-  Remove  counter  recoil  buffer. 

-  Push  cannon  tube  out  of  battery. 

-  Install  counter  recoil  buffer. 

-  Pull  cannon  tube  into  battery. 

-  Fill  and  bleed  replenisher  system. 

-  Fill  out  DA  Forms  2A0A  and  2A07. 


The  first  thing  you  must  do  during  the  task  analysis  review 
is  to  identify  the  natural  breaks  in  the  activity  and  designate 
those  breaks  as  subtasks.  Each  of  the  subtasks  must  define 
an  action  that  has  a  measurable  outcome.  Go  through  the  Job 
Data  Worksheet  (JDW)  for  the  activity  and  designate  the  subtasks. 

Subtask  1  -  _ 


Subtask  2  - 


Subtask  3  - 


Subtask  A  - 


Subtask  5  - 


Subtask  6  - 


For  the  remainder  of  the  review  procedure,  work  with  your 
Subtask  1;  then,  repeat  the  procedure  for  the  remaining  sub¬ 
tasks  you  identified  for  this  activity. 
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Item  2.  Consider  the  following  conditions  under  which  the  subtask 
might  be  performed: 

.  Different  equipment  (M109  SP  Howitzer /MHO  SI’ 

Howitzer) 

.  Number  of  people  performing  the  subtask  (2  during  day/ 

3  at  night) 

.  Equipment  condition  (new  5-ton,  M818/old  5-ton,  M818) 

Are  there  different  conditions  under  which  this  subtask  might 
be  performed? 

Yes _ 

No  _ 

If  Yes,  specify:  _ 


Is  the  subtask  performed  the  same  under  those  different  conditions? 
Yes  _____ 

No 


If  No,  please  specify  which  conditions  and  how  the  subtask  would 
be  done  different.  For  example,  the  task,  "Drive  a  1/4  ton 
truck,"  is  an  obvious  case  of  varying  conditions  affecting  task 
performance.  Drive  how  and  where?  Cross  country?  In  snow? 

On  dry  road?  At  night?  With  blackout?  Any  of  these  conditions 
will  change  the  way  the  task  is  performed. 


Condition 


Difference  in  Subtask 


I 


Item  3.  Does  the  JDK  give  a  complete  listing  of  all  elements 

(decisions  or  actions)  necessary  for  identification  of 
successful  subtask  performance? 

Yes  _ 

No 


If  No,  what  elements 
Added  to  permit 


should  be: 
identification  of 


correct 


performance? 


Deleted  because  not  needed  to  identify  correct  performance? 


Revised  to  permit  identification  of  correct  performance? 
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Item  4.  You  have  seen  this  subtask  performed  many  times.  Of  the  times 
you  have  seen  soldiers  f  a_i_l^  to  perform  it  successfully,  why 
have  they  failed?  Where  do  they  usually  go  wrong?  For  example, 
is  it  because  they  select  the  wrong  tool,  or  use  incorrect 
lubricants?  Please  consider  the  original  list  of  task  elements, 
plus  any  revisions  you  may  have  made,  and  identify  the  element 
that  is  the  most  frequent  cause  of  failure  and  describe  what 
the  soldier  does  wrong. 


Element 


Wha t  Is  It  That  They  Do  Wrong? 


Are  there  other  subtask  elements  that  stand  out  in  your  memory 
as  frequent  causes  of  failure?  If  so,  please  list  them. 


El ement 


What  Is  It  That  Thev  Do  Wrong? 


Item  5.  Sometimes  performance  of  a  subtask  involves  elements  that  are 
very  important  in  the  sense  that  failure  Lo  perform  those 
elements  can  lead  to  serious  consequences,  serious  injury  to 
the  soldier,  or  serious  damage  to  equipment.  Sometimes  this 
is  immediate  and  may  have  irreversible  consequences  (pressing 
the  starter  button  on  a  tank  for  longer  than  15  seconds); 
other  times  it  is  potentially  serious  but  undetected  until 
damage  occurs  (failing  to  tighten  hub  nuts  on  a  1/4  ton  truck 
to  the  specified  torque).  Are  such  elements  part  of  the  sub¬ 
task? 


If  Yes,  identify  them,  describe  what  the  soldier  does  wrong, 
and  indicate  what  the  consequences  are. 

What  Are  the  Consequence* 
Element  What  Is  It  That  Thev  Do  Wrong?  of  Incorrect  _Pe rj/o  r  nyanc ej 


50 


Item  6.  Are  there  groups  of  elements  where  several  sequences  of 
performing  are  possible,  but  only  one  or  some  of  those 
sequences  are  essential  to  correct  performance?  (Do  not 
include  elements  which  are  sequential  because  of  equipment 
design. ] 

Yes _ 

No 


If  Yes,  identify  the  groups  of  elements  and  essential 
sequence(s) : 

Element  Group  Essential  Sequence 


If  they're  done  out  of  sequence,  can  they  be  corrected  at  a 
later  point  without  serious  consequences? 

Yes 


No 


If  No,  which  ones  can  not  be  corrected? 


Item  7 


Is  there  a  necessary  time  standard  for  any  element? 


Yes 


No 


If  Yes,  what  element (s),  and  what  is  the  time? 


Element 


Time 


What  happens  if  he  doesn't  do  it  within  that  time?  For  example, 
if  a  soldier  depresses  the  starter  button  on  a  tank  for  longer 
than  15  seconds,  the  starter  will  burn  out. 


Element 


Wlia t_  Happens 


Now,  return  to  Item  2,  and  repeat  the  review  procedure  for  the 
next  subtask. 
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.SED  T, 
TRAILS] 


B 

,YSIS  FOR 
,  5-TON,  M813" 


A.  Disconnect  power take-of f  shaft  and  PTO  linkage. 

1.  Place  drain  pan  under  vehicle  at  the  transmission  drain  plug. 

2.  Remove  transmission  drain  plug. 

3.  Loosen  set  screw  on  collar  at  splined  end  of  the  propeller  shaft 

4.  Move  collar  away  from  the  yoke. 

5.  Loosen  set  screw  on  front  universal  joint  yoke  at  the  power 
take-off . 

6.  Slide  yoke  from  power  take-off  shaft. 

7.  Remove  cotter  pin  from  shear  pin  at  winch  input  shaft. 

8.  Remove  shear  pin  at  winch  input  shaft. 

9.  Slide  yoke  from  winch  input  shaft. 

10.  Remove  propeller  shaft  from  vehicle. 

11.  Remove  drain  pan. 

.  Transmission  lubricant  must  drain  for  at  least  15  minutes. 

12.  Install  transmission  drain  plug. 

.  Torque  drain  plug  to  60-70  ft/lb. 


B.  Remove  transmission. 


1. 

2. 

3. 

4. 

5. 

6. 

7. 

8. 

10. 

11. 

**12. 

**13. 
**14 . 
**15. 
**16. 
**17. 

**18. 


Remove  16  bolts  and  nuts  securing  transfer  to  transmission 
propeller  shaft. 

Remove  transfer  to  transmission  propeller  shaft. 

Remove  20  screws  and  washers  securing  front  tunnel  and  toeboard 
assembly  to  the  cab  floor. 

Remove  front  tunnel  and  toeboard  assembly. 

Remove  two  capscrews  and  nuts  securing  two  shift  lever  grommet 
clamps . 

Remove  shift  lever  grommet  from  shift  lever  housing  cover. 

Remove  capscrew  securing  gear  shifter  lever  to  shift  lover 
housing  cover. 

Disconnect  main  air  supply  hose,  air  cylinder  to  twin  poppet 
valve  tube,  and  twin  poppet  valve  to  transfer  case  tube  from 
twin  poppet  valve  assembly. 

.  Each  hose  and  tube  must  be  labeled. 

.  Each  hose  must  be  marked  with  a  straight  line  at  the  point 
of  hookup. 

Disconnect  transmission  vent  line. 

Remove  clevis  pin  securing  clutch  actuating  lever  connecting 
link  rod  assembly  to  clutch  release  lower  actuating  lever. 

Loosen  top  two  capscrews  securing  clutch  housing  to  flywheel 
housing . 

.  Two  capscrew's  must  not  be  removed. 

Remove  ten  remaining  capscrews  and  lockwashers. 

Place  chain  around  transmission. 

Position  heavy'  lifting  device  over  transmission. 

Attach  chain  to  heavy  lifting  device. 

Operate  heavy  lifting  device  until  weight  of  transmission  is 
observed . 

Remove  two  capscrews  and  lockwashers  securing  clutch  housing 
to  flywheel  housing. 
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C.  lu-movi-  clutch  assembly. 

1.  Install  clutch  alinement  tool  or  a  transmission  main  drive 
gear  into  hub  of  friction  plate  assembly. 

.  Pressure  plate  assembly  must  be  marked  in  relation 
to  engine  flywheel. 

**2.  Install  three  3/8-16UNC  x  2  1/4  inches  capscrews  and  3/8  x 
1  1/4  inch  flat  washers  in  the  clutch  assembly. 

**3.  Remove  12  capscrews  and  lockwashers  securing  cover  assembly 
to  engine  flywheel. 

.  Capscrews  must  be  turned  one  or  two  turns  in  succession 
to  avoid  distortion  of  cover  assembly. 

4.  Remove  alinement  tool  (transmission  main  drive  gear). 

.  Clutch  assembly  must  be  secured. 

5.  Remove  clutch  assembly  from  engine  flywheel. 
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H.  Test  and  adjust  clutch  assembly  and  inspect  pressure  plat 
for  warpage. 

I.  Check  pressure  plate  for  warpage. 

.  Place  straight  edge  across  pressure  plate.  Is  there 
any  space  between  straight  edge  and  pressure  plate? 

Yes  _  No 

-  If  Yes,  do  the  following: 

.  Note  deficiency  on  DA  Form  2404. 

.  Turn  pressure  plate  into  supervisor. 

-  If  No,  continue  witli  element  2. 

*2.  Adjust  pressure  plate  cover  assembly. 

.  Three  shipping  capscres  are  adjusted  one-by-one  until 
the  pressure  plate  Is  exactly  1  9/32  inches  from  the 
inner  surface  of  the  pressure  plate  cover. 

-  Measurement  is  made  directly  below  shipping  capscrews 

-  Slide  T-rule  is  used  to  measure  distance. 

*3.  Adjust  release  levers. 

.  Lav  clutch  pressure  plate  assembly  face  down  on  level 
surface. 

.  Loosen  lock  nuts  of  the  release  levers. 

.  Turn  the  adjusting  nuts  one-by-one  until  the  top  oi  the 
release  levers  are  exactly  2  5/32  inches  from  the  level 
surface. 

-  Slide  T-ruie  is  used  to  measure  distance. 

.  Tighten  lock  nuts. 
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E.  Install  clutch  assembly. 


1.  Install  two  alincmcnt  pins  in  holes  where  cover  assembly  is 
secured  to  engine  flywheel. 

2.  Position  engine  flywheel,  friction  plate,  and  cover  assembly 
in  flywheel  housing. 

.  Place  cover  assembly  over  two  alineraent  pins. 

.  Do  not  ge t  grease  on  any  parts  of  clutch. 

*3.  Aline  engine  flywheel  and  friction  plate. 

.  Engine  flywheel  and  friction  plate  must  be  alined  using 
a  clutch  alinement  tool  or  a  transmission  main  drive  gear. 

4.  Install  ten  capscrews  and  lockvashers. 

5.  Remove  two  alinement  pins. 

6.  Install  two  capscrews  and  lockwashers. 

7.  Tighten  each  capscrew  alternately. 

.  Capscrew's  must  be  tightened  alternately  until  clutch 
cover  assembly  is  seated  evenly  to  engine  flywheel. 

S.  Torque  capscrews  to  28-32  ft/lb,  evenly  and  alternately. 

9.  Remove  clutch  alinement  tool  (transmission  main  drive  gear). 

10.  Remove  three  3/8-16  UNC  x  2  1/4  inches  capscrews  and  3/8  >:  1 
inches  flat  washers. 


1/4 
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1  .  Install  tran.smi.ssiDD. 


] . 

In 

sta 

1  i  two 

alinement 

pins  in 

top  of 

f lywhee 

1 

housi ng 

■1 

hi 

f  t 

transmi 

ssion. 

3. 

SI 

i  de 

transm 

ission  input  shaft 

up  to 

clutch 

di 

sc  hub. 

A. 

A1 

i  ne 

spline 

s  of  input 

shaft  t 

o  clutch  disc 

liu 

b. 

Did 

spline 

s  aline? 

Yes 

No 

-  If  No,  do  the  following: 


.  Insert  shifter  lever  in  shifter  housing  cover. 
.  Place  transmission  in  fourth  or  fifth  gear. 

.  Turn  output  shaft  slightly  to  aline  splines. 

.  Continue  with  element  5. 


-  If  Yes,  continue  with  element  5. 


5.  Slide  transmission  forward  over  two  alinement  pins  in  top  of 
flywheel  housing. 

6.  Install  ten  capscrews  ana  lockwashers . 


7.  Remove  two  alinement  pins. 


8. 

9. 

10. 


Install  two  caps 
Torque  capscrews 
Install  transfer 
.  Slip  yoke  must 


crews  and  lockwashers. 
to  105-120  ft/lb. 
to  transmission  propeller  shaft, 
be  toward  the  source  of  power. 


.  Heads  of  bolts  must  be  on  outside  of 
transmission  propeller  shaft. 


transfer 


to 


11.  Connect  transmission  vent  line. 

12.  Put  gear  shifter  lever  and  retainer  in  place. 

13.  Install  capscrew  which  secures  gear  shifter  lever  to  shift  lever 
housing  cover. 

19.  Connect  main  air  supply  hose,  air  cylinder  to  twin  poppet  valve 
tube,  and  twin  poppet  valve  to  transfer  case  tube  to  twin  poppet 
valve  assembly. 


Each  hose  must  he  connected  to  the  correct  openings  on 
twin  poppet  valve  assembly. 
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15.  Put  shift  lever  grommet  in  place  on  shift  lever  housing 

lb.  Install  two  capscrews  and  nuts  securing  two  shift  lever 
grommet  clamps. 

17.  Install  front  tunnel  and  toeboard  assembly. 


cover . 


61 


Connect  power  take-off  shaft  and  PTC)  linkage. 

Put  slip  yoke  end  of  propeller  shaft  on  power  take-off  shaft. 

.  Yoke  must  be  put  over  the  key. 

.  Set  screw  must  be  tightened  sufficiently  to  secure 
yoke  to  power  take-off  shaft. 

Put  front  universal  joint  yoke  on  winch  input  shaft. 

.  Shear  pin  must  be  put  through  yoke  and  winch  input 
sliaf  t . 

.  Shear  pin  must  be  secured  with  cotter  pin. 

Move  collar  toward  rear  universal  joint  yoke. 

.  Collar  must  be  moved  until  3/4  inch  exists  between 
collar  and  universal  joint  yoke. 

.  Collar  set  screw  must  be  tightened. 

Fill  transmission  with  lubricant  in  accordance  with  L09-2320- 
260-12. 

.  Transmission  must  be  filled  with  19  pts.  of  lubricating 
oil,  Gear  (GO). 

-  GO  grade  as  follows: 

Expected  Temperature-  Above  +32°F  +40°F  to  -10°F  0°F  to  -65°F 

Grade  GO  90  GO  80  GOS 
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H.  Adjust  clutch  linkage  and  free  travel. 

I.  Replace  clevis  pin  securing  clutch  actuating  lever  connecting 
link  rod  assembly  to  clutch  release  lower  actuating  lever. 

2.  Mark  1  1/2  to  2  inches  on  the  clutch  pedal. 

.  Mark  must  be  made  1  1/2  to  2  inches  below  pad  on 
clutch  pedal . 

3.  Depress  clutch  pedal.  Clutch  should  begin  to  disengage  when 
clutch  pedal  is  depressed  to  the  pencil  mark.  Did  it? 

Yes _ _ _  No  _ 

.  If  Yes,  you  are  finished. 

.  If  No,  continue  with  element  4. 

4.  Remove  clevis  pin  securing  clutch  actuating  lever  connecting 
link  rod  assembly  to  clutch  release  lower  actuating  lever. 

5.  Loosen  connecting  link  clevis  lock  nut. 

6.  Turn  clevis  on  the  connecting  link  rod  as  necessary  to  obtain 
correct  free  travel. 

.  Clevis  must  be  turned  counterclockwise  to  increase  free 
travel . 

.  Clevis  must  be  turned  clockwise  to  decrease  free  travel. 

7.  Replace  clevis  pin  securing  clutch  actuating  lever  connecting 
link  rod  assembly  to  clutch  release  lower  actuating  lever. 

*8.  Depress  clutch  pedal.  Is  free  travel  correct?  Y’es _ No 

.  If  Yes,  you  are  finished. 

.  If  No,  repeat  elements  4-8. 
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APPENDIX  C 

TASKS  AND  TASK  ELEMENTS  FROM  ARTS  FOR  SME  REVIEW 


TASK  It  1  Adjust  transmission  linkage  on  M113A1  APC  (6311) 


1.  Loosen  locknut  on  range  selector  link  at  cross  shaft  end. 

2.  Remove  screw  and  nut  securing  range  selector  link  to  cross 
shaf  t . 

3.  Place  range  selector  lever  in  neutral  position. 

4.  Full  shift  arm  up  to  end  of  travel,  then  back  one  detent  to 
place  transmission  in  neutral  position. 

5.  Adjust  range  selector  link  to  free  pin  fit  at  cross  shaft  arm. 

6.  Adjust  range  link  at  each  range  selector  position  to  obtain 
positive  transmission  detents  in  all  positions. 

7.  Install  screw  and  nut  to  secure  range  selector  link  to  cross 
shaft. 

8.  Tighten  locknut  on  range  selector  link. 
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TASK  f‘2  Inspect  M35A2  electrical  system  (6311) 

Six  malfunctions  installed  in  electrical  system: 

1.  Right  front  turn  signal  light  inoperative. 

2.  Oil  pressure  gage  inoperative. 

3.  Battery  to  battery  cable  loose. 

4.  Low  air  pressure  warning  buzzer  inoperative. 

3.  Generator/alternator  drive  belts  loose. 

6.  Cannon  plug  on  rear  of  main  light  switch  loose. 


1.  Operate  turn  signals. 

2.  Record  "inoperative  right  front  turn  signal,"  or  words  to  that 
effect,  on  DA  Form  2404. 

3.  Inspect  instrument  cluster  with  vehicle  engine  running. 

4.  Record  "oil  pressure  gage  inoperative,"  or  words  to  that  effect, 
cn  DA  Form  2404. 

5.  Record  "low  air  pressure  warning  buzzer  inoperative,"  or 
words  to  that  effect,  on  DA  Form  2404. 

6.  Stop  vehicle  engine  upon  noting  zero  oil  pressure  reading. 

7.  Inspect  batteries  and  battery  box. 

8.  Record  "battery  to  battery  cable  loose,"  or  words  to  that 
effect,  on  DA  Form  2404. 

9.  Inspect  light  switch. 

10.  Record  "cannon  plug  on  rear  of  light  switch  loose,"  or  words 
to  that  effect,  on  DA  Form  2404. 

11.  Inspect  charging  system. 

12.  Record  "generator  (alternator)  drive  belts  loose,"  or  words 
to  that  effect,  on  DA  Form  2404. 
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TASK  it 3  Adjust  cam  dwell  on  M151A1/A2  truck  (63H) 


1.  Remove  igniter  plug  on  top  of  distributor. 

2.  Insert  igniter  adapter. 

3.  Connect  white  lead  from  dwell  meter  to  adapter. 

4.  Disconnect  if  1  spark  plug  cable  adapter. 

3.  Connect  spark  plug  cable  adapter. 

6,  Connect  spark  plug  lead  from  tach  to  adapter. 

7,  Connect  dwell  meter  battery  lead  to  battery  (positive  first) 

8,  Set  RPM  scale  on  dwell  meter  to  5000. 

9,  Set  dwell  meter  cylinder  selector  to  4. 

10.  Operate  vehicle  engine  at  600  RPM. 

11.  Switch  RPM  scale  on  dwell  meter  to  1000  after  engine  reaches 
600  RPM. 

12.  Adjust  points  until  dwell  meter  reads  39°  -  44°. 


TASK  ?;4  Adjust  clutch  cover  assembly  on  M809  series  truck  (63H) 

1.  Locate  correct  page  and  paragraph  in  TM. 

2.  Place  clutch  cover  assembly  on  edge  of  flat  surface. 

3.  Adjust  shipping  capscrews  one-by-one  until  pressure  plate 
is  exactly  1  9/32  inches,  measured  directly  below  shipping 
capscrews,  from  inner  surface  of  pressure  plate  cover  flange. 

4.  Place  clutch  cover  assembly  face  down  on  flat  surface. 

5.  Turn  adjusting  nuts  one-by-one  until  top  of  the  release 
levers  are  exactly  2  5/32  inches  from  the  flat  surface. 

6.  Tighten  locknuts  over  adjusting  nuts. 

7.  Recheck  measurements. 
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TASK  ?;5  Test  and  adjust  alternator  voltage  output  on  M151A1/A2 
truck  (6311) 

1.  Make  a  visual  inspection. 

2.  Test  batteries  for  proper  charge. 

3.  Perform  alternator  output  test. 

4.  Adjust  alternator  voltage  output.* 

5.  Remove  pipe  plug  from  the  front  flange  (pipe  plug  might  be 
at  the  rear) . 

6.  Adjust  alternator  voltage  output  control  until  voltage  read 
exactly  28  volts.* 

7.  Replace  pipe  plug. 


*  These  elements  are  identical  and  were  treated  as  Element  6 
during  all  the  analyses. 
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true 


'I ASK  •'-'()  Remove  and  replace  trout  ditierential  on  MlMAl/AJ 
(Mil) 

1.  Jack-up  both  front  wheels. 

2.  1'lace  jack  stands  to  support  each  wheel. 

i .  Remove  differential  flange  guard. 

•1 .  Remove  drive  shaft  l:-  joint  from  differential  drive  flange 
on  both  left  and  right  sides. 

1.  Remove  1 ront  propeller  shaft  front  differential  drive  flange, 
ft.  Remove  differential  assembly  from  front  crossmember. 

7.  Mount  differential  assembly  to  front  crossmember. 

8.  Install  front  propeller  shaft  on  differential  drive  flange. 

9.  Install  both  left  and  right  drive  shaft  U-joints  on 
differential  drive  flange. 

10.  Replace  differential  flange  guard. 
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TASK  Troubleshoot  25  AMPKKK  DC  charging  syst  i-ia  ((>3L) 


The  task  is  to  diagnose  a  problem  in  the  charging  system  of 
an  M151A1.  The  problem  is:  The  operator  complains  that  tin- 
batteries  are  being  overcharged  and  water  has  to  he  added 
to  the  batteries  at  an  abnormally  high  frequency.  The 
batteries  can  be  assumed  to  be  OK. 


1.  Install  the  generator  regulator  adapter. 

2.  Open  the  link  on  the  generator  regulator  adapter. 

3.  Connect  LVCT  leads  to  LVCT. 

4.  Connect  LVCT  leads  to  adapter  and  ground. 

5.  Idle  engine  at  1000-2000  RPM  (approx,). 

6.  Read  voltage  with  range  selector  in  50  volt  position. 

7.  Correctly  interpret  reading  (regulator  faulty,  voltage 
too  high). 

8.  Turn  off  voltmeter  and  discontinue  test. 
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TASK  r2  Troubleshoot  start  in}'  system  circuit  on  Ml  MAI  /A  2  truck 
(03C) 

The  task  is  to  diagnose  a  problem  in  the  starting  system  of 
an  M151A1.  The  vehicle  sometimes  cranks  slowly.  Results 
from  performing  a  starter  voltage  test  indicate  that  there 
is  excessive  resistance  somewhere  in  Liu-  starting,  circuit. 
You  must  find  the  cause  ol  the  resistance.  The  starter  and 
batteries  have  been  replaced  and  can  he-  considered  to  be  OK. 


1.  Visually  check  all  connections. 

2.  Connect  LVCT  voltmeter  leads  for  performing  a  battery  ground 
cable  test. 

3.  Place  LVCT  voltmeter  range  selector  in  tiie  50  volt  position. 

4.  Crank  engine  with  ignition  switch  turned  off  while  observing 
vo. I  tmeter . 

5.  With  engine  cranking,  progressively  select  lower  voltmeter 
ranges  until  a  reading  is  obtained  or  the  1  volt  range  is 
reached . 

6.  Correctly  interpret  voltmeter  reading  (battery  ground  circuit 
OK)  . 

7.  Connect  LVCT  voltme  ter  leads  for  performing  a  battel  y  to 
battery  cable  test. 

8.  Place  LVCT  voltmeter  range  selector  in  the  50  volt  position. 

9.  Crank  engine  with  the  ignition  switch  turned  off  while  ebservin 
voltmeter . 

10.  With  engine  cranking,  progressively  select  lower  voltmeter 
ranges  until  a  reading  is  obtained  or  until  the  1  volt  range 
is  reached. 

11.  Correctly  interpret  voltmeter  reading  (excessive  resistance 
in  battery  to  battery  '•able). 
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TASK  ii 3  Replace  steering  linkage  on  M151A1/A2  (63C) 

1.  Remove  idler  arm  bracket  from  frame. 

2.  Remove  idler  arm  bracket  and  bushing  from  idler  arm. 

3.  Remove  idler  arm  from  idler  arm  rod  assembly. 

4.  Install  idler  arm  in  rod  assembly  turning  iL  in  until  all 
threads  are  completely  engaged  and  then  backing  the  idler 
arm  out  1  1/2  turns. 

5.  Thread  idler  arm  bracket  onto  idler  arm  until  all  threads 
are  engaged  and  then  back  bracket  off  1  1/2  turns, 

6.  Secure  idler  arm  bracket  to  frame  and  Lorque  Lo  25-35  lb  ft. 

7.  Remove  lubrication  fitting  and  torque  idler  arm  bushing  to 

idler  arm  rod  to  100-110  lb  ft. 

8.  Remove  lubrication  fitting  and  torque  idler  arm  bushing  to 
idler  arm  bracket  to  100-110  lb  ft. 
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1  ASK  it 4 


Troubleshoot  brakes  and  controls  on  the  M151A1/A2  (63C) 


The  task  is  to  detect  the  malfunctions  in  the  brake  system 
of  an  M151A1  and  note  them  on  the  DA  Form  2404.  The  vehicle 
is  prepared  as  follows: 

1.  Left  front  wheel  and  drum  removed. 

2.  Retracting  spring  disconnected. 

3.  Brake  pedal  free  travel  out  of  adjustment  (excessive). 

4.  Low  master  cylinder  fluid  level. 

5.  Right  rear  wheel  cylinder  leak. 

6.  Parking  brake  out  of  adjustment. 

1.  Inspect  left  front  wheel  brake  assembly  and  detect  the  disconnected 
brake  retracting  spring. 

2.  Check  the  action  of  the  brake  pedal  and  determine  that  the 
free  travel  was  out  of  adjustment. 

3.  Inspect  the  master  cylinder  and  note  that  the  fluid  level  was 
too  low. 

4.  Inspect  the  individual  wheel  assemblies  and  detect  that  the  right 
rear  wheel  cylinder  was  leaking. 

5.  Chock  the  action  of  the  parking  brake  and  determine  that  it  was 
maladjusted. 
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TASK  f?5  Troubleshoot  CD850  transmission  (M60A1 )  (63C) 

The  task  is  to  diagnose  and  correct  a  problem  in  the  CD850 
transmission.  The  proglem  is  this:  The  vehicle  drives  in 
reverse,  creeps  backward  in  neutral,  and  stalls  when  shifted 
to  low  or  high. 

1.  Check  shift  linkage  adjustment  by  placing  transmission  shift 
control  in  neutral  and  checking  to  see  that  shift  position 
indicator  on  transmission  points  to  neutral. 

2.  Determine  that  shift  linkage  was  properly  adjusted. 

3.  Remove  right  side  brake  adjustment  access  cover. 

4.  Remove  reverse  range  adjusting  screw  lockplate. 

3.  Loosen  reverse  range  adjusting  screw  locknut  while  holding 
adjusting  screw  witli  wrench. 

6.  Back  off  locknut  far  enough  to  prevent  false  torque  reading. 

7.  Tighten  adjusting  screw  to  50  lb  ft. 

8.  Back  off  adjusting  screw  5  to  6  flats  to  the  nearest  flat  that 
will  align  with  lockplate  when  installed. 

9.  Put  scribe  or  pencil  marks  on  adjusting  screw  and  transmission 
case . 

10.  While  holding  adjusting  screw  to  prevent  it  from  turning, 
tighten  locknut  to  150  lb  ft.  (Repeat  procedure  if  marks  become 
misaligned . ) 

11.  Install,  lockplate  and  transmission  access  plate. 

12.  Drive  vehicle  forward  and  backward  to  verify  that  the  band 
adjustment  was  the  problem  and  that  the  problem  has  been 
corrected . 


TASK  >76  Adjust  shift  control  linkage  (M60A1)  (630) 


1.  Place  transmission  shift  control  lever  in  neutral. 

2.  Check  position  indicator  on  transmission  control  valve  body 
and  determine  that  linkage  adjustment  was  required. 

3.  Disconnect  shift  linkage  at  clevis  on  control  valve  body. 

4.  Insert  locating  pin  through  clevis  and  bracket  closest  to 
shift  control  lever  (View  "A,"  pg.  2-303)  and  determine  that 
linkage  adjustment  was  OK  at  this  point. 

5.  Insert  locating  pin  through  clevis  and  bracket  (bolted  to 
right  side  of  transmission  (View  "E,"  pg.  2-304)  and  determine 
that  linkage  adjustment  up  to  that  point  was  OK. 

6.  Try  to  reconnect  linkage  rod  end  at  clevis  on  valve  body 
(View  "F,"  pg.  2-304),  check  for  free  pin  fit,  and  determine 
that  linkage  should  be  adjusted  at  this  point. 

7.  Adjust  linkage  at  shift  position  indicator  on  control  valve 
body  (View  "F,"  pg.  2-304)  so  free  pin  fit  is  obtained  with 
position  indicator  in  neutral  position. 

8.  Remove  all  locating  pins. 
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APPENDIX  D 

DATA  MATRIXES  FOR  ARTS  TASKS 
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Test  and  ndiust  alternator  voltage  output  on  M151A 
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APPENDIX  E 

MEANS  AND  STANDARD  DEVIATIONS  FOR  ARTS  TASKS 


Table  E-l 


Means  and  Standard  Deviations  for  63H  Task  it  1, 
Adjust  Transmission  Linkage  on  M113A1  APC 


ELEMENT 

MEAN 

STANDARD  DEV 

CASES 

TASK  TOTAL 

7.1192 

1.1940 

151 

1 

0.6225 

0.4864 

151 

2 

0.9470 

0.2247 

151 

3 

0.9338 

0.2495 

151 

4 

0.8278 

0.3788 

151 

5 

0.9669 

0.1795 

151 

6 

0.8742 

0.3328 

151 

7 

0.9868 

0.1147 

151 

8 

0.9603 

0.1960 

151 

Table  E-2 


1 

1 


Means  and  Standard  Deviations  for  63H  Task  // 2, 
Inspect  M35A2  Electrical  System 


ELEMENT 

MEAN 

STANDARD  DEV 

CASES 

TASK  TOTAL 

5  .0926 

1  .5500 

108 

1 

0  .8241 

0.3825 

108 

2 

0  .7500 

0  .4350 

108 

3 

0  .6481 

0  .4798 

108 

4 

0  .6944 

0  .4628 

108 

5 

0  .5278 

0  .5016 

108 

6 

0  .8241 

0  .3825 

108 

7 

0  .8241 

0  .3825 

108 

Task  E-3 


Means  and  Standard  Deviations  for  63H  Task  //3, 
Adjust  Cam  Dwell  on  M151A1/A2  Truck 


ELEMENT 

MEAN 

STANDARD  DEV 

CASES 

TASK  TOTAL 

10  .7333 

2  .9786 

90 

1 

0  .7667 

0  .4253 

90 

2 

0  .8333 

0  .3748 

90 

3 

0  .9000 

0  .3017 

90 

4 

0  .8222 

0  .3845 

90 

5 

0  .7778 

0  .4181 

90 

6 

0  .7889 

0  .4104 

90 

7 

0  .7444 

0  .4386 

90 

3 

0  .7889 

0.4104 

90 

9 

0.8556 

0.3535 

90 

10 

0.8667 

0.3418 

90 

11 

0.8889 

0.3160 

90 

12 

0.9000 

0.3017 

90 

13 

0.8000 

0.4022 

90 

93 


Table  E-4 


Means  and  Standard  Deviations  for  63H  Task  #4, 
Adjust  Clutch  Cover  Assembly  on  M809  Series  Truck 


ELEMENT 

MEAN 

STANDARD  DEV 

CASES 

TASK  TOTAL 

5.9241 

1  .2276 

79 

1 

0.9494 

0.2206 

79  • 

2 

0.6835 

0.4681 

79 

3 

0.8101 

0.3947 

79 

4 

0  .9747 

0.1581 

79 

5 

0  .6456 

0.4814 

79 

6 

0  .9494 

0.2206 

79 

7 

0.9114 

0.2860 

79 

94 


Table  E-5 


Means  and  Standard  Deviations  for  63H  Task  it 5, 

Test  and  Adjust  Alternator  Voltage  Output  on  M151A1/A2  Truck 


ELEMENT 

MEAN 

STANDARD  DEV 

CASES 

TASK  TOTAL 

5  .0667 

1  .3955 

105 

1 

0  .5714 

0  .4972 

105 

2 

0.5524 

0.4996 

105 

3 

0  .4571 

0.5005 

105 

4 

0  .6667 

0  .4737 

105 

5 

0  .9238 

0  .2666 

105 

6 

0.9619 

0.1923 

105 

7 

0.9333 

0.2506 

105 

95 


Table  E-6 


Means  and  Standard  Deviations  for  63ri  Task  #6, 

Remove  and  Replace  Front  Differential  on  M151A1/A2  Truck 


ELEMENT 

MEAN 

STANDARD  DEV 

CASES 

TASK  TOTAL 

9  .8333 

0.5817 

132 

1 

0.9848 

0.1226 

132 

2 

1.0000 

0.0000 

132 

3 

0.9848 

0.1226 

132 

4 

0.9621 

0.1916 

132 

5 

0.9773 

0.1496 

132 

5 

0.9848 

0.1226 

132 

8 

0.9924 

0.0870 

132 

9 

0.9924 

0.0870 

132 

10 

0.9621 

0.1916 

132 

11 

0.9924 

0.0870 

132 

Tabic  E-7 


Means  and  Standard  Deviations  for  63C  Task  it  1, 
Troubleshoot  25  Ampere  DC  Chargins  System 


ELEMENT 

MEAN 

STANDARD  DEV 

CASES 

TASK  TOTAL 

5.5532 

1 .3490 

94 

1 

0.4468 

0.4998 

94 

2 

0.6809 

0.4686 

94 

3 

0.4574 

0.5009 

94 

4 

0.5957 

0.4934 

94 

5 

0.8191 

0.3870 

94 

6 

0  .7553 

0  .4322 

94 

7 

0  .8298 

0  .3778 

94 

8 

0  .9681 

0  .1767 

94 

97 


Table  E-8 


Means  and  Standard  Deviations  for  63C  Task  #2, 
Troubleshoot  Starting  System  Circuit  on  M151A1/A2  Truck 


ELEMENT 

MEAN 

STANDARD  DEV 

CASES 

TASK  TOTAL 

8  .7822 

1  .8687 

101 

1 

0  .8416 

0.3670 

101 

2 

0  .5446 

0.5005 

101 

3 

0  .8812 

0  .3252 

101 

4 

0  .9109 

0  .2863 

101 

5 

0  .7921 

0  .4078 

101 

6 

0  .7525 

0  .4337 

101 

7 

0  .7822 

0  .4148 

101 

8 

0  .8812 

0  .3252 

101 

9 

0  .9010 

0  .3002 

101 

10 

0  .8515 

0  .3574 

101 

11 

0  .6436 

0  .4813 

101 

Table  E-9 


Means  and  Standard  Deviations  for  63C  Task  // 3, 
Replace  Steering  Linkage  on  M151A1/A2  Truck 


ELEMENT 

MEAN 

STANDARD  DEV 

CASES 

TASK  TOTAL 

7. 6667 

1.  0429 

96 

1 

0. 8958 

0.3071 

96 

2 

0.  9792 

0.1436 

96 

3 

0.9688 

0.1749 

96 

4 

0.9792 

0.1436 

96 

5 

0.9896 

0.1021 

96 

6 

0.9271 

0.2614 

96 

7 

0.9583 

0.2009 

96 

8 

0.9688 

0.1749 

96 

Table  E-10 


Means  and  Standard  Deviations  for  63C  Task  #4, 
Troubleshoot  Brakes  and  Controls  on  M151A1/A2  Truck 


ELEMENT 

MEAN 

STANDARD  DEV 

CASES 

TASK  TOTAL 

3.5630 

1 .2315 

135 

1 

0.8222 

0.3837 

135 

2 

0.6519 

0.4782 

135 

3 

0.8000 

0.4015 

135 

4 

0.5481 

0  .4995 

135 

5 

0  .7407 

0  .4399 

135 

100 


Table  F-ll 


Means  and  Standard  Deviations  for  63C  task  // 5, 
Adjust  Shift  Control  Linkage,  M60A1 


TASK  TOTAL 
1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 


10  .0305 

1  .8353 

131 

0.7634 

0  .4267 

131 

0  .8626 

0  -3456 

131 

0  .7786 

0  ,4168 

131 

0.9695 

0  ,1727 

1'3 1 

0.9008 

0-3001 

131 

0  .9237 

0.2666 

131 

0  .8321 

0  .3752 

131 

0.7863 

0  .4115 

131 

0  .5420 

0  .5001 

131 

0  .8321 

0  .3752 

131 

0  .9618 

0  .1923 

131 

0  .8779 

0  .3287 

131 

Table  E-12 


Means  and  Standard  Deviations  for  63C  Task  if 6, 
Adjust  Shift  Control  Linkage,  M60A1 


ELEMENT 

MEAN 

STANDARD  DEV 

CASES 

TASK  TOTAL 

6  .4531 

1  .6403 

128 

1 

0  .8594 

0.3490 

128 

2 

0  .6484 

0  .4793 

128 

3 

0  .8438 

0.3645 

128 

4 

0.6328 

0.4839 

128 

5 

0.7266 

0.4475 

128 

6 

0.9063 

0.2926 

128 

7 

0.9219 

0.2694 

128 

8 

0.9141 

0.2814 

128 

102 


APPENDIX  F 

REGRESSION  SUMMARY  SCORES  FOR  ARTS  TASKS 


Table 


Unstandardized  regression  coefficient. 
Standardized  regression  coefficient. 


Regression  and  Summary  Scores  for  63H 


and  Summary  Scores  for  63H  Task  #5 


Standardized  regression  coefficient. 


Regression  and  Summary  Scores  for  63H  Task  // 6, 
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Standardized  regression  coefficient. 


Table 


Unstandardized  regression  coefficient. 
Standardized  regression  coefficient. 


Table  F-10 


114 


Standardized  regression  coefficient. 


Regression  and  Summary  Scores  for  63C  Task  #6 
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